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EXECUTIVE  SUMMARY 


STUDY ;  An  Evaluative  Study  of  the  K’vy  Medical  Department's 

Patient  Classification  and  Staffing  Allocation 
System  (The  Workload  Management  System  for  Nursing) 

Final  Report  -  Research  Report  5-85 

INVESTIGATORS:  CDR  Karen  A.  Rieder,  NC.  USN 

CDR  Susan  S.  Jackson,  NC,  USNR-R 
Research  Department 
Naval  School  of  Health  Sciences 
Bethesda,  Maryland  20814-5033 

PURPOSE:  The  purpose  of  this  study  was  to  evaluate  the  validity  and 

reliability  of  the  Workload  Management  System  for  Nursing  (WMSN)  which 
includes  a  patient  classification  system  and  staffing  methodology.  The 
system  is  currently  being  used  in  36  naval  hospitals.  In  addition,  the 
perceptions  of  registered  nurses  regarding  the  usefulness  of  the  system  as  a 
management  tool  were  measured  using  written  questionnaires. 

POPULATION:  The  WMSN  was  tested  at  six  hospitals  selected  to  provide  a 

representative  mix  of  CONUS  facilities  by  size,  geographic  location,  nursing 
unit  configuration,  and  population  served.  Additional  selection  criteria 
included  the  availability  of  up-to-date  monthly  nurse  staffing  information 
and  inter-rater  reliability  testing  reports  for  a  minimum  of  four  months. 

METHOD  OF  DATA  COLLECTION:  At  each  study  site  approximately  20%  of  the 
inpatient  census  (n  =  229)  was  randomly  selected  from  the  ICU,  CCU,  Peds, 
Nursery,  and  Medical-Surgical  units  for  reliability  testing.  (The  Psych  tool 
which  was  under  development  during  this  study  was  not  included.)  The 
patients  selected  for  inclusion  in  the  study  were  classified  using  the 
Patient  Classification  Critical  Indicator  tool  by  both  the  investigator  and 
by  a  registered  nurse  assigned  to  the  unit.  The  classifications  were 
completed  independently  within  two  hours  of  each  other.  To  evaluate 
staffing  adequacy,  charge  nurses  completed  a  questionnaire  at  the  end  of 
each  shift  for  a  three  day  period.  To  evaluate  perceptions  of  the  quality 
of  direct  and  indirect  care  given  during  the  three  day  test  period,  charge 
nurses  and  staff  nurses  completed  a  Nursing  Care  Evaluation  Form.  A 
demographic  questionnaire  to  characterize  the  respondents  and  obtain 
feedback  on  perceptions  of  strengths,  weaknesses,  and  usefulness  of  the 
system  was  also  obtained. 

In  addition,  congruent  validity  of  the  Patient  Classification  tool  was 
determined  during  a  parallel  study  conducted  by  the  Army  at  five  Medical 
Treatment  Facilities.  Using  the  Nursing  Care  Hours  Standards  (NCHS)  tool 
developed  by  the  Army,  which  had  content  related  and  criterion  related 
validity,  selected  patients  were  classified  by  Army  investigators  using  both 
tools.  To  ensure  utility  of  the  findings,  the  Navy  and  Army  investigators 
established  their  inter-rater  reliability  with  the  WMSN  Patient 
Classification  tool  prior  to  initiating  data  collection. 


METHOD  OF  ANALYSIS:  For  inter-rater  reliability,  an  agreement  level  of  80 
percent  was  set  as  the  minimally  acceptable  criterion.  Agreement  was  sought 
between  categories  and  between  factors.  The  relationship  between  category 
scores  was  tested  using  the  Kappa  Statistic.  Analysis  of  variance 
Intra-Class  Correlation  (ICC)  was  used  to  demonstrate  agreement  within 
factors.  Validity  was  established  by  testing  the  relationship  between  the 
WMSN  classification  and  the  NCHS  classification  tooU  using  the  Pearson 
Product  Moment  (PPM)  Correlation.  Descriptive  statistics  were  used  to 
determine  level  of  user  acceptability  and  perceptions  of  system  strength  and 
weaknesses.  Analysis  of  variance  and  Scheffe  comparisons  were  used  to 
compare  charge  nurses'  perceptions  of  staffing  adequacy  and  perceptions  of 
quality  care  given  under  various  staffing  conditions  for  452  shifts.  The 
same  analyses  were  used  to  ascertain  the  relationship  between  staffing 
conditions  and  the  performance  of  direct  and  indirect  care  activities. 

FINDINGS:  The  inter-rater  reliability  agreement  level  for  the  six 

hospitals  was  85  percent.  To  rule  out  consistent  variation,  the  Kappa 
Statistic  was  calculated  and  found  to  be  .78.  Category  agreement  was 
significant  at  the  pc  .01  level.  Within  specialty  units  category  agreements 
were:  ICU/CC  =  94%,  Medicine  =  76%,  Nursery  =  96%,  Pediatrics  =  84%, 
Post-Partum  =  88%,  and  Surgery  =  88%  with  all  Kappa  Statistics  significant 
at  pc.Ol.  Inter-rater  reliability  within  factors  on  the  Patient 
Classification  Critical  Indicators  Instrument  was  calculated  using  the  ICC. 
Findings  for  all  factors  were  above  .90  except  for  the  emotional  support, 
complex  treatments,  teaching,  and  simple  treatments  factors.  All 
correlations  were  significant  at  pc.Ol.  The  PPM  correlation  between  the 
Army's  NCHS  tool  and  the  WMSN  patient  classification  tool  based  on  a  random 
sample  of  141  patients  was  .81  (Army,  1984). 

Of  434  nurse  respondents,  78.6%  were  staff  nurses,  12.2%  were  charge 
nurses,  and  9.2%  were  supervisors.  Of  this  sample  81.7%  worked  in  two  large 
hospitals.  The  WMSN  system  was  rated  as  "moderately"  to  "very  easy"  to  use 
by  90.4%,  was  perceived  as  being  "usually"  or  "always"  accurate  by  50.7%, 
and  as  "useful"  or  "very  useful"  by  49%  of  the  sample;  74%  were  "neutral"  to 
"very  satisfied"  with  the  system.  Of  the  nurses  responding  to  the 
satisfaction  questions,  81%  indicated  that  they  had  not  seen  the  daily 
summary  sheets  or  monthly  staffing  graphs.  (Thus,  their  perceptions  were 
based  solely  on  the  patient  acuity  portion  of  the  system.)  Perceptions  of 
satisfaction  varied  significantly  for  those  who  had  and  those  who  had  not 
been  involved  in  staffing  decisions  (F  =  29.07,  pc  .01). 

Usefulness  as  a  management  tool  and  ease  of  use  were  identified  as  the 
major  strengths  of  the  system.  Major  weaknesses  were  inaccuracy  in 
reflecting  workload,  lack  of  comprehensiveness,  and  lack  of  reliability  from 
staff  member  to  staff  member.  Of  those  respondents  who  expressed  dissatis¬ 
faction  with  the  accuracy  and  comprehensiveness  of  the  system,  most  worked 
in  the  L&O  or  psychiatric  specialty  areas.  Given  a  choice,  51%  of  the  staff 
nurses,  82%  of  the  charge  nurses,  and  88%  of  the  supervisory  nurses  would 
continue  to  use  the  present  system.  These  responses  were  positively 
correlated  with  having  seen  the  WMSN  staffing  summary  sheets  and  graphs. 

Over  half  of  the  respondents  stated  that  it  takes  less  than  two  minutes  to 


classify  a  patient.  Ninety-five  percent  prefer  to  classify  only  once  a  day, 
and  that  classification  is  to  be  on  the  day  shift  {83X). 

Shift  staffing  adequacy  was  obtained  from  charge  nurses'  subjective 
reports.  Across  the  hospitals,  87  out  of  452  or  19.2%  of  the  shifts  were 
identified  as  being  staffed  poorly  or  less  than  adequately.  Factors 
influencing  staffing  patterns  consisted  of  greater  than  usual  number  of 
patients  requiring  either  extensive  nursing  care  time  (22.6%)  or  special 
treatments  or  procedures  (18.8%),  and  less  than  adequate  number  of  staff  on 
duty  (19.2%).  When  asked  what  would  have  helped,  90.8%  of  the  charge  nurses 
indicated  that  additional  staff  was  needed,  with  45%  citing  that  an 
additional  nurse  would  have  solved  the  problem.  Charge  nurses'  perceptions 
of  quality  of  care  provided  and  perceptions  of  staffing  adequacy  were 
significantly  correlated  (r(452)  =  .71,  p<.01). 

To  evaluate  the  relationship  between  actual  staffing  and  charge  nurses' 
perceptions  of  staffing  adequacy  and  perceptions  of  quality  of  care  given 
during  the  452  shifts,  a  Workload  Index  was  created  using  reconmended 
staffing  as  determined  by  the  WMSN.  The  means  for  perceptions  of  staffing 
adequacy  and  perceptions  of  quality  of  care  provided  were  compared  to  the 
Workload  Index  (+2  staff,  il  staff,  and  -2  staff  levels)  using  the  Scheffe 
method.  Results  of  the  multiple  comparisons  revealed  significant  (p<.05) 
differences  between  the  three  staffing  levels  for  charge  nurses'  perceptions 
of  how  adequately  a  unit  was  staffed  and  for  perceptions  of  the  quality  of 
care  provided.  In  addition,  significant  positive  linear  relationships  were 
found  between  the  Workload  Index  and  perceptions  of  quality  care  (r  *  .28, 
p  <  .01)  and  staffing  adequacy  (r  =  .24,  p<..01). 

Charge  nurses  (n  =  468)  and  staff  nurses  (n  =  464)  evaluated  the 
quality  of  direct  and  indirect  care  provided  on  each  shift  using  a  Nursing 
Care  Evaluation  Questionnaire.  The  internal  consistency  (Cronbach's  Alpha) 
for  both  subscales  on  this  questionnaire  was  .98.  Perceptions  of  quality 
care  provided  differed  between  the  two  groups  for  vital  signs  (F  =  5.91, 
p<.01)  and  for  administrative  duties  (F  =  4.67,  p<.03).  In  each  case, 
staff  nurses  evaluated  the  care  given  as  better  than  the  charge  nurses.  Two 
components  of  direct  care  judged  as  being  provided  less  than  adequately  by 
both  groups  were  teaching  U7%)  and  emotional  support  (21%).  Indirect  care 
activities  judged  as  completed  less  than  adequately  by  a  large  percentage  of 
nurses  were:  a)  initiating  and  updating  the  patient  care  plans  (40%);  b) 
performing  administrative  duties  (33%);  c)  making  rounds  with  physicians 
(39%);  d)  allowing  for  personal  time  to  include  meals  and  breaks  (28%);  and 
e)  orienting  new  personnel  (24%). 

To  ascertain  the  relationship  between  actual  staffing  levels  and 
perceptions  of  how  "well"  direct  and  indirect  care  activities  were 
performed,  registered  nurse  staffing  on  452  shifts  was  categorized  into 
three  levels  using  the  Workload  Index  criteria.  Results  of  an  ANOVA  to 
compare  the  composite  direct  and  indirect  care  sub'^cales  mean  values 
revealed  significant  differences  in  the  values  (p^.Ol)  across  all  Index 
levels.  To  determine  if  the  mean  values  of  the  individual  nursing  factors 
(groups  of  activities)  within  the  subscales  differed  based  on  the  Workload 
Index,  separate  analyses  were  conducted.  For  both  direct  care  and  indirect 
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care  activities,  the  mean  values  differed  significantly.  Multiple  mean 
comparisons  revealed  that  alj.  means  for  the  factors  in  both  subscales  were 
different  (p<:.05)  except  for  feeding,  which  differed  only  if  the  RN  staff 
was  below  recommended  (-2  or  more). 

CONCLUSIONS;  Study  results  indicated  that  the  Workload  Management  System 
for  Nursing  patient  acuity  tool  is  both  valid  (as  measured  by  the  NCHS  tool) 
and  reliable.  Factors  identified  as  having  a  low  intra-class  correlation 
will  be  made  explicit  to  hospitals,  so  that  appropriate  teaching  and 
monitoring  regarding  documentation  can  be  ensured.  Nurses  perceived  the 
system  as  useful  for  management  purposes,  and,  in  general,  were  satisfied 
with  the  system.  This  perception  of  satisfaction  was  significantly  affected 
by  whether  the  nurse  was  involved  with  entire  system  or  only  the  patient 
classification  portion,  and  by  the  specialty  unit  on  which  the  subjects 
worked. 

Charge  Nurses'  perceptions  of  quality  of  care  given  and  staffing 
adequacy  were  significantly  related  to  staffing  levels  as  defined  by  the 
WMSN.  A  significant  finding  was  the  relationship  between  actual  staffing 
(as  defined  by  the  Workload  Index)  and  nurses'  perceptions  of  how  well 
direct  and  indirect  nursing  care  were  provided.  Although  classification 
systems  have  been  judged  as  extremely  effective  in  matching  workload  to 
numbers  of  staff,  the  linkage  between  quantity  of  nursing  personnel  to 
quality  of  outcome  has  been  elusive.  The  results  of  this  study  demonstrate 
that  quality,  as  defined  by  perceptions  of  how  "well"  nursing  care  was 
provided,  is  linearly  related  to  the  numbers  of  staff  available  to  give  that 
care  on  a  nursing  unit.  Therefore,  these  results  give  some  credence  to  the 
assumptions  that  quantity  and  category  of  staff  (i*e.,  RN)  are  directly 
related  to  quality  of  care  provided. 

Though  the  Workload  Index  revealed  that  only  13%  of  the  452  shifts  were 
understaffed  by  -2  personnel  or  more,  several  indirect  care  activities  were 
judged  as  completed  inadequately  by  a  large  percentage  of  respondents.  A 
parallel  study  conducted  by  Misener  and  Freline  (1983)  reported  that,  across 
nine  Army  hospitals,  average  time  spent  by  direct  nursing  care  providers  was 
distributed  as  28.5%  for  direct  care,  56.5%  for  indirect  care  and  15%  as 
unavailable  for  patient  care.  These  percentages  were  validated  during  the 
re-analysis  of  a  study  conducted  by  Kelly  (1980)  in  three  naval  hospitals 
and  results  reported  for  the  civilian  community  (Lake,  1982).  The  tested 
WMSN  system  did  not  include  unavailable  time  and  allocated  only  45-65%  of 
staff  time  for  indirect  care  depending  upon  hospital  type  and  room 
configuration.  A  requirements  model  predicated  on  the  current  WMSN 
percentages  does  not  appear  to  give  adequate  time  for  indirect  care 
functions.  Based  on  these  findings,  the  percentages  allotted  to  direct 
care,  indirect  care,  and  unavailable  time  for  patient  care  by  specialty  area 
were  realigned  and  incorporated  into  the  current  system. 

RECOMMENDATIONS;  As  a  result  of  the  study,  recommendations  were  made  for: 
ongoing  validity  and  reliability  assessments;  extending  computerization  of 
the  system  to  all  hospitals;  development  of  a  mark-sense  version  of  the 
patient  classification  tool;  and  extending  the  system  to  cover  the 
measurement  of  patient  care  requirements  in  Labor  and  Delivery,  Recovery 
Room,  and  Ambulatory  Care. 

xiv 


ACKNOWLEDGEMENTS 


The  authors  wish  to  thank  the  many  individuals  who  were  instrumental  in 
the  successful  completion  of  this  study.  Foremost,  we  wish  to  thank  the 
nursing  staff  and  directors  of  nursing  services  at  the  six  study  sites. 

Their  assistance  and  support  were  vital  for  ensuring  the  validity  and 
accuracy  of  the  findings.  We  are  also  indebted  to  RADM  Mary  Nielubowicz,  NC, 
USN,  Director,  Navy  Nurse  Corps,  for  her  support  and  encouragement  in 
completing  this  project.  Special  appreciation  is  extended  to  members  of  the 
Nursing  Research  Service  at  Walter  Reed  Army  Medical  Center  for  their 
collaboration. 

We  are  indebted  to  the  staff  of  the  Naval  School  of  Health  Sciences  for 
their  suggestions,  patience,  and  moral  support.  Terrence  Kay,  Mathematical 
Statistician,  provided  guidance  on  statistical  instruments;  Lieutenant 
Timothy  Steele,  MSC,  USNR,  Research  Associate,  served  as  consultant  on 
statistical  analysis;  and  Dale  Edgeington,  Computer  Programmer,  and  Karen 
Washington,  Data  Transcriber,  assisted  with  the  computer  programming.  Judy 
Emmons,  Secretary,  was  responsible  for  typing  the  final  report. 

Finally,  we  are  especially  grateful  to  Captain  Phyllis  J.  Elsass,  NC, 
USN,  Commanding  Officer,  Naval  School  of  Health  Sciences,  for  her  excellent 
suggestions  and  organizational  support  and  to  the  Naval  Medical  Research  and 
Development  Command  for  the  funding  to  conduct  this  research. 


AUTHORS 


Connander  Karen  A.  Rieder,  NC,  USN,  is  principal  investigator  and 
project  direction  for  this  study.  During  the  conduct  of  this  study, 

CDR  Rieder  was  Director,  Research  Department,  Naval  School  of  Health 
Sciences,  Bethesda,  Maryland. 

Commander  Susan  Jackson  conducted  the  study  to  meet  the  requirements  of 
HSA  270:  Research  in  Health  Services  Administration,  under  the  direction  of 
Dr.  R.  F.  Southby,  Ph.D.,  in  partial  fulfillment  of  the  requirements  for  a 
Post-Masters  Graduate  Certificate  in  Health  Services  Administration  from  The 
George  Washington  University,  Washington,  DC. 
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1.  INTRODUCTION  AND  PURPOSE. 

The  purpose  of  this  study  was  to  evaluate  the  validity  and  reliability  of 
the  Workload  Management  System  for  Nursing  (WMSN)  which  had  been  under 
development  since  1981.  In  addition,  the  perceptions  of  registered  nurses 
regarding  the  usefulness  of  the  system  as  a  management  tool  were  measured 
using  written  questionnaires.  A  preliminary  report  published  in  August  1984 
presented  the  background  information  for  the  study,  including  the  hypotheses, 
literature  review,  instrumentation,  and  analysis  of  the  results  from  the 
first  test  site  (Rieder  &  Jackson,  1984).  This  final  report  summarizes  and 
analyzes  the  data  from  the  six  naval  hospitals  included  in  the  study.  To 
simplify  the  reading  and  interpretation  of  the  results,  the  definition  of 
terms  and  hypotheses  will  be  repeated  in  this  report. 

2.  DEFINITION  OF  TERMS. 

Workload  Management  System  for  Nursing:  a  systematic  process  for 
determining  staffing  requirements  based  upon  identified  patient  care  needs. 
The  system  includes  a  patient  classification  tool  and  a  staffing  methodology. 
The  patient  classification  instrument  is  of  factor  evaluative  design  and 
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requires  that  a  registered  nurse  assess  ten  factors  related  to  direct  patient 
care  and  assign  a  score  to  each  factor.  The  assessment  consists  of  both 
retrospective  and  prospective  components;  that  is,  assessment  of  care 
received  and  required  during  the  day  shift  is  used  to  predict  care 
requirements  for  the  next  24  hours.  The  weighted  factor  scores  are  summed 
resulting  in  the  patient  being  classified  into  one  of  six  discrete 
categories.  The  staffing  methodology  is  designed  to  determine  the  actual 
nursing  care  hours  required  for  a  specified  group  of  patients  and  the  numbers 
and  mix  of  personnel  recommended  to  provide  quality  care.  This  system 
incorporates  both  direct  and  indirect  care  time. 

Patient  classification:  the  grouping  of  patients  according  to  an 
assessment  of  their  nursing  care  requirements  over  a  specified  period  of 
time. 

Critical  Indicators:  those  nursing  activities  on  the  patient 
classification  instrument  that  have  the  greatest  impact  on  direct  care  time. 

Factors ;  a  group  of  critical  indicators  that  cover  one  specific  domain 
of  activities.  They  include  ten  areas:  vital  signs,  monitoring,  activities 
of  daily  living,  feeding,  simple  treatments/procedures/medications,  complex 
treatments/procedures/medications,  respiratory  therapy,  IV  therapy,  teaching, 
and  emotional  support. 

Points :  the  values  assigned  to  each  specific  critical  indicator  based 
upon  documented  time  and  motion  studies.  Each  point  is  equal  to  7h  minutes 
of  direct  nursing  care  time. 

Category:  the  representative  grouping  of  patients  according  to  their 
nursing  care  requirements.  The  Workload  Management  System  consists  of  six 
categories.  A  Category  I  patient  requires  minimal  care  whpreas  a  Category  : 


patient  requires  intensive  care,  that  is,  more  than  one  staff  member  to  one 
patient  relationship. 


Direct  Nursing  Care  Time:  the  activities  that  take  place  in  the  presence 
of  the  patient  and/or  family.  These  activities  are  observable,  behavioral, 
and  include  the  following:  placement  of  equipment  at  bedside,  explanation  of 
procedure  to  patient,  preparation  of  patient,  performance  of  treatment, 
removal  of  equipment  from  area,  recording  of  treatment  at  bedside, 
assessment/observation  of  patient  response,  and  teaching. 

Indirect  Nursing  Care  Time:  those  activities,  conditions,  and 
circumstances  that  necessitate  time  over  and  above  direct  care.  To  address 
these  factors,  indirect  care  time  and  special  allowances  have  been 
incorporated  into  the  nursing  care  hour  requirements  for  each  of  the  six 
patient  care  categories.  During  this  evaluative  study,  an  indirect  care  time 
percentage  ranging  from  45  to  75  percent  was  built  into  the  nursing  care  hour 
requirements  for  each  category.  The  percentages  were  distributed  as  follows: 
indirect  care  plus  unpredicted  needs  -  45%,  teaching  hospital  allowance  - 
10%,  and  semi-private  room  allowance  -  20%. 

Nursing  Care  Hour  Requirements:  the  hours  of  ncrsing  care  time  required 
for  each  category  of  patient  based  upon  an  assessment  of  their  direct  and 
indirect  nursing  care  needs.  This  is  operationalized  via  six  pre-calculated 
nursing  care  hour  requirement  charts  which  incorporate  two  factors:  type  of 
unit  (open,  semi-private  room,  nursery,  or  light  care)  and  type  of  facility 
(teaching  vs.  non-teaching  hospital). 

Personnel  Requirements:  the  number  and  mix  of  RNs  (Registered  Nurses) 
and  NRNs  (Non-Registered  Nurses)  required  to  care  for  the  patient  workload  on 


a  unit.  This  is  operationalized  via  two  charts:  acute  care  and  intensive 
care.  The  acute  care  chart  allocates  a  40X  RN  to  60%  NRN  personnel  mix  and 
distributes  45%  of  staff  to  the  day  shift,  35%  to  the  evening  shift,  and  20% 
to  the  night  shift.  In  contrast,  the  intensive  care  chart  utilizes  a  60%  RN 
to  40%  NRN  personnel  mix  which  is  evenly  distributed  across  all  shifts. 

RN:  a  professional  Registered  Nurse  who  has  satisfactorily  completed  an 
orientation  program  to  the  hospital. 

NRN:  nursing  personnel  other  than  RNs  who  have  satisfactorily  completed 
an  orientation  program  to  the  hospital.  These  include  Hospital  Corps 
personnel,  LPNs,  and  ward  clerks. 

Workload  Index:  a  comparison  between  the  actual  nursing  staff  on  a  unit 
and  the  number  recommended  by  the  WMSN  based  upon  patient  requirements.  For 
this  study  the  index  levels  were  defined  as  less  than  recommended  (-2  staff 
or  more),  at  recommended  (-1  staff),  and  greater  than  recommended  (+2  staff 
or  more). 

Trained  Rater:  a  nurse  (RN)  who  has  undergone  standardized  training  in 
the  use  of  the  patient  classification  instrument. 

Inter-Rater  Reliability:  level  of  agreement  (in  factors  and  in 
categories)  achieved  when  two  trained  raters  independently  assess  a  group  of 
patients  on  a  specified  unit  during  the  same  time  period  using  the  patient 
classification  instrument.  An  80%  agreement  level  is  required. 

Intra-Service  Reliability:  level  of  agreement  in  factors  and  in 
categories  within  each  clinical  service  (medicine,  surgery,  pediatrics, 
ICU/CCU,  nursery,  and  post-partum)  when  trained  raters  independently  classify 
patients  using  the  patient  classification  instrument.  An  30%  agreement  level 
is  required. 


3.  METHODOLOGY. 

a.  Hypotheses.  In  order  to  determine  the  validity  and  reliability  of 
the  WMSN  for  quantifying  inpatient  care  workload  and  for  establishing 
manpower  requirements,  the  following  hypotheses  were  tested: 

1.  The  WMSN  patient  classification  tool  will  have  high  (80% 
agreement)  inter-rater  reliability  for  patient  category  at  each 
test  facility. 

2.  The  WMSN  patient  classification  tool  will  have  high  (80% 
agreement)  intra-service  reliability  for  patient  category  on  like 
specialty  nursing  units  across  the  test  facilities. 

3.  The  WMSN  patient  classification  tool  will  exhibit  high  internal 
consistency  within  its  factors. 

4.  Validity  of  the  WMSN  Patient  Classification  tool  will  be 
established  by  a  high  correlation  with  the  Nursing  Care  Hours 
Standards  instrument  which  possesses  content  and  criterion-based 
validity. 

5.  Nurses  will  express  more  satisfaction  than  dissatisfaction  with 
the  WMSN. 

6.  There  will  be  a  positive,  significant  relationship  between  charge 
nurses'  perceptions  of  staffing  adequacy  and  quality  of  care 
given  and  the  Workload  Index. 

7.  There  will  be  a  positive,  significant  relationship  between  charge 
and  staff  nurses'  perceptions  of  how  well  direct  and  indirect 
care  activities  were  provided  to  patients  and  the  Workload  Index. 
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b.  Setting  and  Selection  Criteria.  Over  a  six  month  period,  the  WMSN 
was  evaluated  at  six  study  sites  selected  to  provide  a  representative  mix  of 
CONUS  naval  hospitals  by  bed  size,  geographical  location,  nursing  unit 
configuration,  and  mission  to  population  served.  Additional  criteria 
included  the  availability  of  up-to-date  monthly  nurse  staffing  summary 
information  and  reports  of  inter-rater  reliability  testing  using  the  patient 
classification  instruments  for  a  minimum  of  four  months.  The  six  naval 
hospitals  included  in  the  study  were:  1)  two  small  hospitals  -  NH  Cherry 
Point  and  NH  Lemoore;  2)  two  medium  hospitals  -  NH  Charleston  and  NH  Camp 
Pendleton;  and  3)  two  large  teaching  hospitals  -  NH  Oakland  and  NH 
Portsmouth . 

A  detailed  review  of  the  procedure,  copies  of  the  data  collection 
instruments,  and  a  description  of  the  statistical  tests  nerformed  can  be 
found  in  the  preliminary  report. 

c.  Instruments.  At  the  six  study  sites  approximately  20%  of  the 
inpatient  census  (n  =  229)  on  35  nursing  units  were  randomly  selected  from 
ICU/CCU,  pediatrics,  nursery,  post-partum,  medical,  and  surgical  units  for 
reliability  testing.  The  patient  sample  was  classified  using  the  Patient 
Classification  Critical  Indicator  Instrument  by  both  the  investigator  (nurse 
expert)  and  a  registered  nurse  classifier  on  each  nursing  unit. 

Validity  of  the  Patient  Classification  tool  was  determined  during  a 
parallel  study  conducted  by  the  Army  at  five  Medical  Treatment  Facilities 
(Norton,  1984).  Seltcted  patients  were  classified  by  Army  investigators 
using  both  the  WMSN  Patient  Classification  tool  and  the  Army  Nursing  Care 
Hours  Standards  (NCHS)  tool,  which  had  established  content  and 
criterion-related  validity  (Sherrod,  Rauch,  &  Twist,  1981).  To  ensure 
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generalizability  of  the  findings,  the  Navy  and  Army  investigators  established 
their  inter-rater  reliability  with  the  WMSN  Patient  Classification  tool  prior 
to  initiating  data  collection. 

To  characterize  the  respondents  and  to  obtain  feedback  on  perceptions  of 
the  strengths,  weaknesses,  and  usefulness  of  the  WMSN,  a  demographic 
questionnaire  was  completed  by  all  participants.  To  measure  perceptions  of 
staffing  adequacy  and  perceptions  of  the  quality  of  nursing  care  given, 
charge  nurses  on  all  shifts  over  a  three  day  period  (n  =  452)  conpleted  the 
Unit  Staffing  Evaluation  Questionnaire.  These  subjective  findings  were  then 
correlated  with  the  actual  number  and  mix  of  personnel  assigned  to  the  ward 
and  the  recommended  number  and  mix  of  staff  proposed  by  the  WMSN  system. 

Lastly,  all  nurses  (charge  and  staff)  were  requested  to  indicate  their 
perceptions  of  how  well  direct  nursing  care  was  given  and  indirect  nursing 
tasks  were  completed  on  each  of  nine  shifts.  These  perceptions  were  obtained 
through  administration  of  the  Nursing  Care  Evaluation  Questionnaire.  To 
ascertain  if  perceptions  differed  based  on  role,  the  responses  of  the  charge 
nurses  (n  =  468)  and  staff  nurses  (n  =  464)  were  compared. 

d.  Method  of  Analysis.  For  inter-rater  reliability,  an  agreement  level 
of  80  percent  was  set  as  the  minimally  acceptable  criterion.  Agreement  was 
sought  between  the  six  patient  categories  and  between  the  ten  critical 
indicator  factors.  The  relationship  between  category  scores  was  tested  using 
the  Kappa  Statistic.  An  analysis  of  variance  (ANOVA)  intra-class  correlation 
(ICC)  approach  was  selected  to  examine  rater  agreement  within  the  factors 
(Ventura,  Hageman,  Slakler,  &  Fox,  1980).  Congruent  validity  was  established 
by  testing  the  relationship  between  the  WMSN  Classification  and  the  NCHS 
Classification  tools  using  the  Pearson  Product  Moment  (PPM)  Correlation. 


Descriptive  statistics  were  used  to  determine  the  level  of  user  acceptability 
and  perceptions  of  system  strengths  and  weaknesses.  Analysis  of  variance  and 
Scheffe  comparisons  were  used  to  compare  the  charge  nurses'  perceptions  of 
staffing  adequacy  and  perceptions  of  quality  of  care  given  under  various 
staffing  conditions  for  452  shifts.  The  same  analyses  were  performed  to 
ascertain  the  relationship  between  staffing  conditions  and  the  performance  of 
direct  and  indirect  nursing  care  activities. 


4.  PRESENTATION  AND  ANALYSIS  OF  DATA. 

a.  Description  of  Subjects.  The  study  sample  consisted  of  434 
registered  nurses  from  six  naval  hospitals.  The  group  was  comprised  of  78 
males  (18%)  and  356  females  (82%)  with  56  percent  being  under  34  years  of 
age.  By  nursing  position  78.6%  were  staff  nurses,  12.2%  were  charge  nurses, 
and  9.2%  were  supervisors,  administrators,  and  educators.  Of  this  sample, 
81.7%  worked  in  the  two  large  teaching  hospitals.  All  nu'*ses  were  familiar 
with  the  Navy  Workload  Management  System,  but  12%  indicated  that  they  did  not 
classify  patients  on  a  regular  basis.  A  demographic  profile  of  participants 
from  each  naval  hospital  is  presented  in  Appendices  Al-Fl. 

Using  the  hypotheses  as  a  framework,  the  following  results  were  obtained 
from  the  study  data. 


b.  Composite  Findings  on  the  Reliability  of  the  Workload  Management 


System  For  Nursing.  A  comparison  of  the  patient  category  agreement  (n  =  229) 
between  the  nurse  expert  and  charge  nurse  classifiers  for  35  nursing  units  at 
six  naval  hospitals  is  presented  in  Table  1.  The  inter-rater  reliability 


category  agreement  across  six  hospitals  was  85  percent.  To  rule  out 
consistent  variation,  a  Kappa  Statistic  was  calculated  and  found  to  be  .78 


TABLE  1 


PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BETWEEN  NURSE  EXPERT  AND  CHARGE  NURSE  CLASSIFIERS 
ACROSS  SIX  NAVAL  HOSPITALS 


CBARCB  NURSES  BATIHG 

NURSE  FREQUENCY  I 

expert  percent  I 


RATING  RUU  PCT 

COL  per 

1  Category 

1  1 

Category 

1  2 

Category 

1  3 

Category 

1  4 

—  —  -  — 

Category 

i  5 

1 

TOTAL 

.  .  . 

■■IT 

■ 

»♦ 

i 

1 

42 

1 

9 

1 

0 

1 

0 

0 

51 

1 

la.34 

1 

3.93 

i 

0.00 

1 

0.00 

0.00 

1 

22.27 

Category 

1 

82.35 

1 

17.65 

i 

0.00 

0.00 

0.00 

93.33 

9.];3.  1 

0.00 

0.00 

1 

0.00 

1 

9' 

2 

3 

1 

85 

f 

15 

1 

0 

1 

0 

f 

103 

L.3i 

37.12 

6.55 

1 

O.QO 

» 

0.00 

44.98 

Category 

1 

2.91 

82.52 

14.56 

I 

0.00 

0.00 

1 

1 

6.67 

86.73 

1 

22.73 

0.00 

0.00 

1 

AemMseMiMiM 

•  .  • 

•-  • 

>^mm 

3 

1 

0 

1 

.  4 

51 

2 

1 

0 

1 

57 

0*00 

1 

L.75 

22.27 

1 

0.87 

0.00 

1 

24.89 

Category 

0.00 

7.02 

I 

89.47 

1 

3.51 

0.00 

1 

1 

0.00 

1 

4.00 

1 

77.27 

14.29 

0.00 

1 

4 

1 

0 

1 

-  0 

1 

0 

12 

1 

1 

1 

13 

1 

0.00 

1 

0.00 

t 

0.00 

1 

5.24 

0.44 

1 

5.68 

1 

0.00 

0.00 

1 

0.00 

92.31 

7.69 

1 

Category 

1 

0.00 

0.00 

0.00 

85.71 

1 

16.67 

1 

5 

1 

0 

i 

0 

0 

1 

0 

5 

1 

5 

1 

0.00 

0.00 

0.00 

0.00 

1 

2.  IB 

1 

2.18 

Category 

1 

0.00 

1 

0.00 

1 

0.00 

1 

0.00 

1  100.00 

1 

-  .  . 

1 

0.00 

1 

O.QO 

1 

o 

•  • 

8 

1 

0.00 

1 

83.33 

1 

TOTAL 

45 

98 

66 

14 

6 

229 

19.65 

42.79 

28.82 

6.11 

2.62 

100.00 

Percent  of  Agreement  -  195/229  or  85.15X 
Kappa  Statistic  «  .784 
Standard  Deviation  of  Kappa  ■  .034 
Z  Score  ■  22.9^9 

Category  agreement  significant  at  p  < .001  using  Kappa  Statistic. 


which  was  significant  at  p-c.OOl  level.  Table  2  summarizes  the  agreement 
levels  and  Kappa  Statistics  for  each  study  hospital.  A  breakdown  of  the 
reliability  results  by  category  for  the  individual  study  sites  can  be  found 
in  Appendices  A2-F2. 

To  compare  the  reliability  of  the  VWSN  across  specialty  services,  percent 
agreement  and  Kappa  Statistics  were  analyzed  for  the  nursery,  pediatric, 
post-partum,  critical  care,  medical,  and  surgical  units  (Table  3).  The 
highest  level  of  agreement  (96%)  occurred  in  the  nursery  units  and  the  lowest 
(76%)  in  the  medical  units  with  Kappa  Statistic  significant  (p^.OOl)  for  all 
specialty  areas.  Detailed  tables  of  category  classification  agreement 
between  the  nurse  expert  and  charge  nurses  for  each  specialty  service  can  be 
found  in  Appendices  Gl-66. 

c.  Composite  Findings  on  the  Reliability  (Internal  Consistency)  of  the 
Factors  Within  the  Patient  Classification  Instrument.  Internal  consistency 
within  the  ten  factors  of  the  patient  classification  tool  was  analyzed  using 
the  Intra-class  Correlation  Coefficient  (ICC).  ICC  was  chosen  to  estimate 
inter-rater  reliability  in  order  to  rule  out  the  possibility  of  consistent 
variation  between  raters.  As  shown  in  Table  4,  findings  were  above  .90  for 
all  factors  except;  emotional  support  (.50),  complex  treatments  (.58), 
teaching  (.74),  and  simple  treatments  (.74).  All  correlations  were 
significant  at  p^.OOl.  The  ICC  for  total  factor  scores  was  .96.  The  ICC 
findings  for  individual  study  hospitals  are  presented  in  Appendices  A3-F3. 

d.  Validity  of  the  WMSN  Patient  Classification  Instrument.  Professional 
nursing  judgment  was  a  key  factor  in  the  design  of  the  Navy  patient 
classification  tool.  Critical  indicators  were  selected  based  on  input  from 


TABLE  2 


INTER-RATER  RELIABILITY  FOR  PATIENT 
CLASSIFICATION  CATEGORY  AGREEHENT  BETWEEN  NURSE  EXPERT 
AND  CHARGE  NURSES  FOR  SIX  NAVAL  HOSPITALS 


Hospital  Name 

Aqreement  Level 

Kappa  Statistic 

Camp  Pendleton 

88% 

.83 

Charleston 

89% 

.84 

Cherry  Point 

89% 

.82 

Lemoore 

94X 

.86 

Oakland 

79% 

.69 

Portsmouth 

81% 

.71 

All  six  hospitals 

85% 

.78 

ALL  Kappa  Statistics  significant  at  p  ^  .001  level 


I 


I 
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TABLE  3 


SUMMARY  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT  BETWEEN 
NURSE  EXPERT  AND  CHARGE  NURSE  CLASSIFIERS  FOR  SPECIALTY  SERVICES 

ACROSS  SIX  NAVAL  HOSPITALS 


Specialty  Area  Percent  of  Agreement  Kappa  Statistic 


Nursery 

(n=26) 

96% 

.93 

Post-Partura 

(n=33) 

88% 

.80 

Pediatric 

(n=25) 

84% 

.74 

ICU/CCU 

(n=18) 

94% 

.92 

Surgical 

(n=52) 

88% 

.88 

Medical 

(n=75) 

76% 

.62 

n  =  number  of  patients  classified  in  a  specialty  service. 
ALL  Kappa  Statistics  significant  at  p  <  .001. 
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TABLE  4 


INTER-RATER  RELIABILITIES  FOR  CRITICAL  INDICATOR  FACTORS 
AS  ESTIMATED  BY  ICC  ACROSS  SIX  NAVAL  HOSPITALS 


Factor 


Intra-Class  Correlation 


Vital  Signs 

.946 

Monitoring 

.967 

Activities  of  Daily  Living 

.953 

Feeding 

.939 

Sinple  Treatments 

.742 

Conplex  Treatments 

.580 

Respiratory  Therapy 

.917 

Intravenous  Therapy 

.927 

Teaching 

.744 

Emotional  Support 

.502 

Continuous  Care 

»  w  w 

TOTAL  POINTS 


.964 
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staff,  charge,  and  supervisory  nurses  at  33  hospitals.  As  each  new  version 
of  the  instrument  was  developed,  its  prima  facie  validity  was  established  by 
clinical  experts  from  various  specialty  services  prior  to  testing. 

Specific  nursing  care  time  assigned  to  each  activity  was  based  upon  the 
results  of  the  four  year  time  and  motion  study  conducted  by  the  Army 
(Sherrod,  et  al.,  1981).  During  this  comprehensive  Nursing  Care  Hour 
Standards  (NCHS)  study,  patient  classification  instruments  for  six  specialty 
areas  were  constructed.  Content  validity  was  ensured  by  input  of 
professional  nurses  during  the  design  and  validation  phases  of  tool 
development.  Validity  of  the  classification  tool  for  predicting  actual 
nursing  requirements  was  established  during  two  independent  testing  periods. 
The  correlation  coefficients  for  the  relationship  between  documented  direct 
nursing  care  requirements  and  the  patient  classification  instruments  ranged 
from  r  =  .98  to  r  =  .99.  Criterion-related  validity  was  established  using 
observational  studies  to  determine  the  relationship  between  actual,  timed 
direct  nursing  care  activities  and  the  instruments.  The  criterion-related 
validity  coefficients  ranged  from  r  =  .87  to  r  =  .99  (Sherrod,  1985). 

Although  the  Navy  had  incorporated  the  Army's  mean  times  for  nursing 
activities  into  its  classification  tool,  the  two  systems  had  been  developed 
independently.  Nursing  activities  were  identified  specific  to  each  military 
service,  and  the  Navy  patient  classification  tool  delineated  indicators  for 
emotional  support  and  patient  teaching.  To  establish  validity  of  the  WMSN 
classification  tool,  10%  of  the  patients  (n  =  141)  at  five  Army  hospitals 
were  randomly  selected  and  independently  classified  by  two  investigators. 

One  investigator  classified  patients  using  the  W^^SN  tool  while  the  other 
rated  the  same  patients  with  the  Army's  NCHS  classification  tool.  To 


decrease  response  bias,  the  two  investigators  alternated  the  tool  used.  The 
Pearson  Product  Moment  correlation  between  the  NCHS  tool  and  the  WMSN  Patient 
Classification  tool  was  .81.  When  the  data  was  adjusted  for  the  emotional 
support  factor  which  had  no  equivalent  in  the  Army  tool,  the  correlation 
increased  to  .89  (Norton,  1984). 

e.  Nurses'  Perceptions  of  Acceptability  and  Satisfaction  With  the 
Workload  Management  System  For  Nursing.  Perceptions  of  satisfaction  with  the 
WMSN  were  obtained  using  information  from  the  Staff  Questionnaire  (n  =  434). 
Perceptions  surveyed  included  ease  of  use,  accuracy,  usefulness,  and  major 
strengths  and  weaknesses  of  the  system.  Data  was  analyzed  using  variables 
such  as  hospital  size,  nursing  position  of  the  respondent,  and  whether  or  not 
nurses  had  seen  the  monthly  graphs  and  daily  staffing  summary  reports.  No 
significant  differences  were  found  between  responses  based  upon  the  size  of 
the  facility  in  which  the  nurses  worked.  However,  there  were  significant 
differences  among  respondents'  perceptions  of  accuracy  of  the  system  in 
reflecting  level  of  care  required,  usefulness  as  a  management  tool,  and 
satisfaction  with  the  WMSN  based  upon  nursing  position  and  whether  or  not  the 
nurse  had  reviewed  the  monthly  staffing  graphs  and  daily  staffing  summary 
sheets. 

The  'WMSN  was  rated  as  "moderately  easy"  to  "very  easy"  to  use  by  90.4%  of 
the  nurses,  was  perceived  as  being  "usually"  or  "always"  accurate  by  50.7%, 
and  as  being  "useful"  or  "very  useful"  by  ^9%.  a  breakdown  by  nursing 
position  showed  that  44%  of  the  staff  nurses,  72.6%  of  the  charge  nurses,  and 
31%  of  the  supervisory  level  nurses  viewed  the  'WMSN  as  "usually"  accurate 
(Table  5).  This  same  pattern  emerged  for  perceptions  of  usefulness.  An 
analysis  by  nursina  position  showed  that  40.2%  of  staff  nurses,  76.5%  of 


TABLE  5 


NURSING 

POSITION 


PERCEPTIONS  OF  ACCURACY  OF  THE  UMSN  IN  REFLECTING 
PATIENT  CARE  REQUIREMENTS  BY  NURSING  POSITION 
ACROSS  SIX  NAVAL  HOSPITALS 


PERCEPTIONS  OF  ACCURACY  OF  WMSN 


FREGUENCr 

ROW  per 

COL  PCT 

■USUALXY  SOMETIMES 

NEVER 

STAFF  NURSE 

1 

147 

1 

177 

1 

10 

i 

1 

44.01 

1 

52.99 

1 

2.99 

1 

1 

68.69 

1 

89.85 

1 

90.91 

1 

•  ▼ 

CHARGE  NURSE 

1 

37 

1 

13 

1 

1 

1 

1 

72.55 

1 

25.49 

1 

1.96 

1 

1 

—  ^  ^ 

17.29 

1 

6  .60 

1 

9.09 

1 

SUPERVISORY 

I 

30 

1 

7 

T  " 

1 

0 

1 

1 

81.08 

1 

18.92 

1 

0.00 

» 

1 

14.02 

1 

3.55 

1 

0.00 

1 

TOTAL 

214 

197 

11 

n  =  422 

Missing  Cases  =  12 
Chi-Square  =  29.586 
Degrees  of  Freedom  =  4 
Probability  =  .0001 


charge  nurses,  and  87.5%  of  supervisory  level  nurses  found  the  WMSN  "useful" 
(Table  6). 

Overall  satisfaction  with  the  system  was  lower  with  36.4%  of  the  nurses 
being  “very  satisfied"  or  "satisfied"  and  38%  selecting  "neutral".  Again, 
there  was  significant  variation  in  perceptions  when  the  data  was  analyzed  by 
nursing  position  and  whether  or  not  the  monthly  staffing  graphs  and  daily 
staffing  summary  sheets  had  been  reviewed.  In  Table  7  the  analysis  by 
nursing  position  revealed  a  significant  pattern  in  which  only  28%  of  the 
staff  nurses  were  “satisfied"  with  the  system  in  contrast  to  73.7%  of  the 
supervisory  level  nurses  who  expressed  satisfaction.  Perceptions  of 
accuracy,  usefulness  and  satisfaction  with  the  WMSN  at  each  individual  test 
site  can  be  found  in  Appendices  A4-F4,  A5-F5,  A6-F6. 

A  closer  analysis  of  the  data  revealed  that  in  four  test  sites  81%  of  the 
respondents,  which  included  96%  of  those  who  were  dissatisfied  (n  *  70),  had 
never  reviewed  the  Daily  Staffing  Sheets  and  Monthly  Staffing  Graphs 
(Table  8).  Their  perceptions  were  based  solely  upon  the  classification 
portion  of  the  WMSN.  Of  the  staff  nurses  in  the  study,  92.3%  (n  =  240)  had 
not  been  privy  to  staffing  information.  This  group  comprised  the  majority 
(89.7%)  of  those  whose  perceptions  of  system  usefulness  were  based  only  on 
the  classification  phase  of  the  system  (Table  9).  The  difference  in 
perceptions  based  upon  exposure  to  the  staffing  portion  of  the  WMSN  is 
compared  in  Tables  10  and  11.  Perceptions  of  satisfaction  varied 
significantly  for  nurses  who  had  and  had  not  been  involved  in  staffing 
decisions  (F  =  29.07,  p  <  .0001). 

The  major  strengths  of  the  system  identified  by  the  respondents  were: 

1)  "usefulness  as  a  management  tool";  2)  "ease  of  use";  and  3)  "tak  is  minimal 


TABLE  6 


PERCEPTIONS  OF  USEFULNESS  OF  THE  WMSN  AS  A  MANAGEMENT  TOOL 
BY  NURSING  POSITION  ACROSS  SIX  NAVAL  HOSPITALS 


OSETULNESS  OF  WMSN  AS  A  MANAGEMENT  TOOL 


NURSING 

POSITION 


FREQUENCY  I 


ROW  PCT 

COL  PCT 

1 

1 

1 

USEFUL 

UNDECIDED 

NOT 

USEFUL 

STAFF  NJRSE 

1 

133 

1 

124 

1 

74 

1 

1 

40.18 

1 

37.46 

1 

22.36 

1 

1 

64.2  5 

1 

89.86 

1 

96.10 

1 

CHARGE  NURSE 

1 

39 

1 

10 

1 

2 

1 

1 

76  .47 

1 

19.61 

1 

3.92 

1 

1 

18.84 

7.25 

1 

2.60 

1 

SUPERVISORY 

1 

35 

1 

4 

f 

I 

1 

1 

87.50 

1 

10.00 

1 

2.50 

1 

1 

18.91 

1 

2.90 

1 

1.30 

1 

TOTAL 

207 

138 

77 

total 

331 


51 


40 


422 


n  =  22 


Missing  Cases  =  12 
Chi-Square  =  50.688 
Degrees  of  Freedom  =  4 
Probability  =  .0001 
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TABLE  7 


NURSING 

POSITION 


SATISFACTION  WITH  THE  WMSN  BY  NURSING  POSITION 
ACROSS  SIX  NAVAL  HOSPITALS 


SATISFACTION  WITH  THE  WMSN 


FPEOUENCY 
ROW  PCT 
COL  PCT 


SATISFIED  NEUTRAL  DISSATISFIED 


- ♦ 


STAFF  NJRSE 

i 

93 

1 

190 

1 

95 

1 

I 

28.35 

1 

92.63 

1 

28.96 

1 

1 

61.18 

1 

88.05 

1 

88.79 

1 

CHARGE  NURSE 

1 

31 

1 

12 

1 

9 

1 

1 

59.62 

» 

23.08 

1 

17.31 

J 

1 

20.39 

1 

7.55 

1 

8.91 

1 

SUPERVISORY 

1 

28 

« 

• 

7 

1 

3 

1 

1 

73  .68 

1 

18.92 

1 

7.89 

1 

18.92 

1 

■ 

9.90 

1 

2.80  1 

TOTAL 

152 

159 

107 

n  *  18 

Missing  Cases  =  16 
Chi-Sqaure  =  44.338 
Degrees  of  Freedom  =  4 
Probability  *  .0001 


TABLE  8 


SATISFACTION  WITH  THE  WMSN  BASED  ON  WHETHER  NURSES  HAD  REVIEWED  OR 
HAD  NOT  REVIEWED  THE  MONTHLY  STAFFING  REPORTS  AND  GRAPHS 
ACROSS  SIX  NAVAL  HOSPITALS 


SATISFACTION 

WITH 

WMSN 


SAW 

MONTHLY 

REPORTS 

FREQUENCY 

I 

ROM  PCT 

x 

COL  PCT 

i 

NO 

YES 

TOTAL 

_  ^  _ 

SATl SFI ED 

1 

73 

1 

35 

”  'r 

1 

108 

» 

67.59 

1 

32.61 

1 

1 

29.67 

1 

60.34 

1 

NEUTRAL 

1 

106 

1 

20 

1 

126 

1 

86.13 

» 

15.37 

1 

1 

>3.09 

1 

36.63 

1 

DISSATISFIED 

1 

67 

1 

3 

•  ^ 

1 

70 

95.71 

1 

6.29 

1 

• 

1 

27.26 

1 

5.17 

1 

TOTAL 

266 

53 

306 

Results  based  on  data  from  four  of  six  study  sites. 
Chi-Square  =  23.188 
Degrees  of  Freedom  =  2 
Probability  =  .0001 


TABLE  9 


COMPARISON  OF  NURSES  WHO  REVIEWED  OR  DID  NOT  REVIEW 
MONTHLY  WMSN  REPORTS  BY  NURSING  POSITION 


Nursing 

Position  Saw  Staffing  Reports/Graphs 


Frequency 

Row  Pet 

Col  Pet 

Saw  Reports 

Did  Not 

See  Reports 

TOTAL 

1 

19 

227 

246 

Staff  Nurse 

7.72X 

92.28X 

32.203C 

89.72t 

1 

19 

1 

17 

36 

Charge  Nurse 

52.78X 

47.22X 

32.20X 

6.72« 

1 

1 

21  i 

7 

30 

Supervisory 

70X  1 

30« 

35.60X 

3.56X 

TOTAL 

59  1 

253 

'  312 

Results  based  on  data  from  four  of  six  hospital  sites. 
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TABLE  10 


SATISFACTION  WITH  WMSN  AMONG  NURSES  WHO  REVIEWED 
MONTHLY  SUMMARY  REPORTS  AND  GRAPHS  BY  NURSING  POSITION 
ACROSS  SIX  NAVAL  HOSPITALS 


Controlling  Variable:  Reviewed  Reports  and  Graphs 


^^URSING 

POSITION 


FREQUENCY 

ROW  per 

COL  PCT 
STAFF  NURSE 


I  SATISFACTION  WITH  WMSN 
j  SATISFIED  NEUTRAL  DISSATISFIED  j 

I  B  I  11  1  0  I 

I  42411  I  57.B9  I  0.00  I 

I  22*86  I  59.00  I  0*00  I 


CHARGE  NURSE  ( 


I  97*89  1  31*58  I  10*53  I 
I  ,  31.43  I  30.00:1  66.67  I 

SUPERVISORY  I  16  I  3  I  II 


,  I  eQ.;00  I  19.00.1  9.00  I 

I  45*71  I  15.00  I  33*33  I 


TOTAL 


TOTAL 

19 

19 

20 

98 


Data  from  four  of  six  hospital  sites. 


TABLE  11 


NURSING 

POSITION 


SATISFACTION  WITH  WMSN  AMONG  NURSES  WHO  HAD  NOT  REVIEWED 
MONTHLY  SUMMARY  REPORTS  AND  GRAPHS  BY  NURSING  POSITION 
ACROSS  SIX  NAVAL  HOSPITALS 


Controlling  Variable:  Did  Not  Review  Reports  and  Graphs 


FREOUEMCY 

SATISFACTION  WITH  WMSN 

ROW  PCT 

CQL  PCT  , 

SATISFIED  NEUTRAL  DISSATISFIED  . 

. . 

1 

1 

TOTAL 

STAFF  MURSE 

1 

59  1  LOO  1 

62  1 

221 

1' 

26«70  1  45.25  1 

28.09  1 

60«:82  1  94«34  ) 
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Data  from  four  of  six  hospital  sites. 


time  to  complete"  (Table  12).  In  analyzing  the  selected  strengths  by  nursing 
position,  there  was  a  significant  difference  between  the  three  roles  for 
usefulness  of  the  system  as  a  management  tool.  This  response  was  selected  by 
33%  of  the  staff  nurses,  60.4%  of  the  charge  nurses,  and  65%  of  the 
supervisory  level  nurses  (Table  13).  The  major  weaknesses  of  the  system  were 
identified  as:  1)  "inaccuracy  in  reflecting  workload";  2)  "lack  of 
comprehensiveness";  and  3)  "lack  of  reliability  from  staff  member  to  staff 
member"  (Table  14).  An  analysis  of  weaknesses  by  nursing  position  revealed 
no  significant  differences.  However,  a  review  of  written  comments  on  the 
questionnaire  indicated  that  the  greatest  degree  of  dissatisfaction  was 
expressed  by  nurses  in  the  labor  and  delivery,  newborn  nursery,  and  ICU/CCU 
specialty  areas.  For  a  list  of  strengths  and  weaknesses  by  individual 
hospitals,  see  Appendices  A7-F7  and  A8-F8. 

Despite  the  weaknesses  identified,  given  a  choice,  51%  of  the  staff 
nurses,  82%  of  the  charge  nurses,  and  88%  of  the  supervisory  nurses  would 
continue  to  use  the  present  system.  Choice  of  responses  was  significantly 
correlated  with  whether  or  not  participants  had  seen  the  WMSN  staffing 
summary  sheets  and  graphs.  Over  half  of  the  respondents  stated  that  it  takes 
less  than  two  minutes  to  classify  a  patient.  Ninety-five  percent  of  the 
respondents  preferred  to  classify  only  once  a  day.  Nurses  working  on  units 
where  patient  care  requirements  fluctuated  dramatically  between  shifts,  such 
as  the  critical  care  units,  requested  an  option  to  classify  more  frequently. 
The  day  shift  was  the  preferred  time  for  classifying  patients  according  to 
33%  of  the  respondents. 

f ,  Correlation  Between  Charge  Nurses'  Perceptions  of  the  Quality  of 
Nursing  Care  Given,  Staffing  Adequacy,  and  Recommended  Staffing  Using  the 
Workload  Index.  Perceptions  of  staffing  adequacy  and  quality  of  nursing  care 
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TABLE  12 


RANK  ORDER  OF  MAJOR  STRENGTHS*  OF  THE  HMSN 
AS  PERCEIVED  BY  NURSES  ACROSS  SIX  NAVAL  HOSPITALS 


Variable 

Agreement 

Frequency 

Percent 

Usefulness  as  Management  Tool 

170 

39.2 

Ease  of  Use 

120 

27.6 

Takes  Minimal  Time  To  Conplete 

91 

21.0 

Reliable 

72 

16.6 

Conprehensive 

68 

15.7 

Accurately  Reflects  Workload 

59 

13.6 

No  Strengths  Noted 

65 

15.0 

n  =®  434  nurses 

*More  than  one  response  could  be  selected. 


TABLE  13 


PERCEPTIONS  THAT  USEFULNESS  AS  A  MANAGEHENT  TOOL 
WAS  A  MAJOR  STRENGTH  OF  THE  UMSN 
BY  NURSING  POSITON 
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Chi-Square  =  26.931 
Degrees  of  Freedom  =  2 
Probability  =  .0001 


TABLE  14 


RANK  ORDER  OF  MAJOR  WEAKNESSES*  OF  THE  UMSN  AS  PERCEIVED 
BY  NURSES  ACROSS  SIX  NAVAL  HOSPITALS 


Variable  Agreement 


Frequency 

Percent 

Inaccurate  in  Reflecting  Workload 

159 

36.6 

Not  Conprehensive 

139 

32.0 

Unreliable 

134 

30.9 

Takes  Long  Time  To  Do 

106 

24.4 

Not  Useful  As  A  Management  Tool 

45 

10.4 

Difficult  To  Use 

19 

4.4 

No  Weaknesses  Found 

22 

5.1 

n  =  434  nurses 


*More  than  one  response  could  be  selected. 
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given  were  obtained  from  charge  nurses  (n  =  452)  who  completed  the  Unit 
Staffing  Evaluation  Questionnaire  at  the  end  of  each  shift  for  three 
consecutive  days.  In  general,  factors  identified  as  influencing  staffing 
patterns  were:  1)  greater  than  usual  number  of  patients  requiring  either 
extensive  nursing  care  time  {22.6%)  or  special  treatments  or  procedures 
(18.9%),  and  2)  less  than  adequate  number  of  staff  on  duty  for  87  out  of  452 
shifts  (19.2%).  When  asked  what  would  have  helped  to  improve  staffing  levels 
on  the  87  understaffed  shifts,  90.8%  of  the  charge  nurses  indicated  that 
additional  staff  was  needed,  with  46%  specifically  requesting  one  additional 
RN  to  solve  the  problem.  In  responding  to  the  single  item  question  about  the 
quality  of  nursing  care  provided  on  the  shift,  charge  nurses  perceived  that 
only  on  25  of  452  shifts  (5.5%)  was  less  than  adequate  care  given.  For  the 
87  shifts  perceived  as  understaffed,  quality  of  care  provided  was  identified 
as  less  than  adequate  24%  of  the  time.  Charge  nurses'  perceptions  of  quality 
of  care  and  staffing  adequacy  were  significantly  correlated  with  perceptions 
of  staffing  adequacy  (r=.71,  p  ^  .0001). 

An  objective  measurement  of  the  hospitals'  staffing  patterns  was  obtained 
using  a  Workload  Index.  Results  of  a  1983  pilot  study  at  two  large  naval 
hospitals  suggested  that  nurses'  perceptions  of  the  quality  of  care  provided 
varied  significantly  when  a  nursing  unit  was  understaffed  by  two  or  more 
persons.  To  test  this  hypothesis,  the  numbers  of  RN,  NRN,  and  total  staff 
assigned  to  each  unit  at  the  study  sites  were  taken  from  the  Workload 
Management  Daily  Staffing  Summary  Sheets.  A  Workload  Index  of  three  levels 
(greater  than  recommended  staffing,  at  recommended  staffing,  and  less  than 
recommended  staffing)  was  developed  for  each  shift  using  the  continuum  from 
plus  two  to  minus  two  persons.  Out  of  544  shifts,  74  shifts  were  identified 
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as  below  the  recommended  staffing  level,  401  shifts  at  the  recommended 
staffing  level,  and  69  shifts  above  the  recommended  staffing  level.  Charge 
nurses  on  452  of  the  544  shifts  (83%)  responded  to  the  quality  and  staffing 
adequacy  questions  and  became  the  sample  for  analysis.  (See  Appendices  A9-F9 
for  Workload  Index  at  each  study  facility.) 

To  evaluate  the  relationship  between  actual  staffing,  perceptions  of 
staffing  adequacy,  and  perceptions  of  quality  of  care  given,  mean  charge 
nurse  perceptions  were  compared  to  the  Workload  Index  ("'‘2  staff,  i-1  staff, 
and  -2  staff  levels)  for  each  shift  using  the  Scheffe  method.  Results  of  the 
multiple  comparisons  revealed  significant  (p  -d  .05)  differences  between  the 
three  staffing  levels  based  on  responses  to  the  single  item  questions  of  how 
well  quality  care  was  provided  (Figure  1)  and  how  adequately  a  unit  was 
staffed  (Figure  2).  In  addition,  significant  positive  linear  relationships 
were  found  between  the  Workload  Index  and  perception  of  quality  care 
(r  =  .28,  p  .01)  and  staffing  adequacy  (r  =  .24,  p  Z  .01). 

g.  Nurses'  Perceptions  of  Direct  and  Indirect  Care  Activities  Performed 
Under  Various  Staffing  Levels.  Both  charge  nurses  (n  =  468)  and  staff  nurses 
(n  =  464)  rated  the  quality  of  direct  and  indirect  nursing  care  provided  on 
each  shift  using  the  Nursing  Care  Evaluation  Form.  This  tool  included  two 
nine  item  subscales:  direct  care  activities  and  indirect  care  activities. 
Using  those  cases  in  which  subjects  responded  to  all  items,  the  internal 
consistency  (Cronbach's  alpha)  for  both  subscales  was  .98  (direct  care 
n  =  408)  (indirect  care  n  =  368).  Perceptions  of  quality  care  provided 
differed  between  the  two  groups  for  vital  signs  (F  =  5.91,  p  ^  .01)  and  for 
administrative  duties  (F  =  4.67,  p  ^.03)  with  staff  nurses  rating  these 
activities  higher  than  charge  nurses. 
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FIGURE  1 


RELATIONSHIP  OF  CHARGE  NURSES*  PERCEPTIONS  OF  QUALITY 
NURSING  CARE  GIVEN  WITH  THE  WORKLOAD  INDEX^ 
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FIGURE  2 


RELATIONSHIP  OF  CHARGE  NURSES'  PERCEPTIONS  OF  STAFFING 
ADEQUACY  WITH  THE  WORKLOAD  INOEX^ 


*An  coitparisons  significant  at  .05  level. 


Key:  -  =  Less  than  reconmended  staffing 
level  (minus  2  staff  or  more). 

OK  =  At  reconmended  staffing  level 
(minus  or  plus  1  staff). 

+  =  Greater  than  reconmended 
staffing  level  (plus  2  staff 
or  more). 
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Each  direct  and  indirect  care  nursing  activity  was  analyzed  by  total 
responses  (n  =  932)  to  detv«rmine  nurses'  perceptions  of  how  "well"  these 
tasks  were  accomplished.  Separate  analyses  of  the  nine  factors  of  direct 
nursing  care  (simple  and  complex  treatments  were  combined  into  one  variable) 
revealed  two  factors  that  were  judged  as  being  provided  "less  than 
adequately"  by  a  relatively  large  percentage  of  nurses.  They  were  teaching 
(17%)  and  emotional  support  (21%).  Indirect  care  components  judged  as 
completed  "less  than  adequately"  by  a  large  percentage  of  nurses  were: 
a)  initiating  and  updating  the  patient  care  plan  (40%);  b)  administrative 
duties  (33%);  c)  rounds  with  physicians  (39%);  d)  personal  time  to  include 
meals  and  breaks  (28%);  and  e)  orienting  new  personnel  (24%). 

In  order  to  ascertain  the  relationship  between  actual  staffing  levels  and 
perceptions  of  how  well  direct  and  indirect  care  activities  were  performed, 
registered  nurse  staffing  on  452  shifts  was  categorized  into  three  levels 
using  the  aforementioned  Workload  Index  criteria.  Results  of  an  ANOVA  to 
compare  the  composite  subscale  mean  values  for  both  direct  and  indirect  care 
(Table  15)  revealed  significant  differences  in  the  values  (p  <  .01)  across 
all  index  levels.  Multiple  comparisons  using  the  Scheffe  procedure 
demonstrated  that  the  composite  subscale  means  were  significantly  different 
from  each  other  (p  <1  .05)  across  all  Index  levels. 

To  determine  if  the  mean  values  of  the  individual  nursing  factors  (groups 
of  activities)  comprising  the  subscales  differed  based  on  the  Workload  Index, 
separate  analyses  were  conducted.  For  both  the  direct  care  (Table  16)  and 
indirect  care  factors  (Table  17),  the  mean  values  were  significantly 
different  (p  <  .01).  Multiple  mean  comparisons  revealed  that  all  means  for 
the  factors  in  both  subscales  differed  significantly  (p  ^  .05)  except  for 
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feeding,  which  differed  only  if  the  RN  staff  was  below  (-2)  recommended. 
These  findings  lend  support  to  the  predictive  validity  of  the  WMSN  patient 
classification  and  staffing  methodology  based  upon  perceptions  of  the 
professional  staff  working  on  the  units  surveyed. 

5.  DISCUSSION. 

A  review  of  reliability  by  category  agreement  across  the  six  facilities 
indicated  that  the  lowest  reliability  existed  within  the  two  teaching 
hospitals.  Although  the  two  facilities  accounted  for  46!t  (n  =  105)  of  the 
total  patients  classified,  a  preponderance  of  the  patients  (63%)  resided  on 
the  medical  units  and  required  complex  treatments  and  procedures  as  well  as 
teaching  and  emotional  support.  As  evidenced  by  a  review  of  the  findings, 
both  the  complex  treatment  and  emotional  support  factors  had  low  intra-class 
correlations.  For  both  factors,  lack  of  appropriate  documentation  appeared 
to  be  the  issue.  Specifically,  the  complexity  of  the  treatments  and  numbers 
of  staff  required  to  provide  that  care  could  not  be  discerned  from  the 
patient  care  plans  or  nurses  notes.  Moreover,  the  nurse  expert  could  not 
always  locate  the  documentation  of  a  patient  problem  requiring  emotional 
support  to  validate  the  points  credited  for  this  factor.  The  measurement 
reliability  in  these  nursing  care  activities  would  probably  have  been  higher 
if  documentation  had  not  been  a  requisite  for  crediting  activities. 

Several  authors  have  indicated  that  the  validity  of  a  system, 
specifically  predictive  validity,  is  a  key  facet  of  any  prospective  staffing 
methodology  based  upon  perceived  nursing  care  requirements  (Giovannetti , 
1979;  Giovannetti  &  Mayer,  1984).  The  finding  that  the  WMSN  direct  care 
Patient  Classification  instrument  correlated  highly  with  the  Nursing  Care 
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Hour  Standards  (NCHS)  instrument,  developed  so  rigorously  by  the  Army,  was 
not  surprising.  Although  the  Army  and  Navy  classification  tools  had  been 
developed  independently,  the  direct  care  time  coefficients  for  each  indicator 
of  the  WMSN  were  derived  from  the  1981  Army  study  and  modified  slightly  based 
on  current  practice,  expert  nursing  judgment,  and  repeated  standard  time 
studies. 

Predictive  validity  for  the  indirect  care  portion  of  the  system  was  a 
more  complex  issue.  Lacking  the  resources  to  conduct  a  rigorous  study  of  how 
nursing  personnel  spend  their  duty  time  across  various  Navy  facilities,  the 
system  developers  had  derived  a  formula  which  distributed  staff  time  across 
direct  and  indirect  care  activities.  The  basis  for  these  percentages  was 
historical  data  and  expert  opinion.  To  determine  if  the  hours  of  care 
accurately  reflected  time  required  to  provide  direct  and  indirect  care, 
charge  nurses'  perceptions  of  staffing  adequacy  on  a  shift  basis  were 
solicited,  as  well  as  nurses'  general  perceptions  of  satisfaction  with  the 
system. 

The  finding  that  charge  nurses'  perceptions  of  staffing  adequacy  and 
quality  of  care  provided  were  significantly  correlated  with  the  staffing 
recommended  by  the  WMSN  gave  initial  credence  to  the  established  hours  of 
care.  However,  further  analyses  revealed  that  the  indirect  care  time  built 
into  the  system  may  be  inadequate.  Though  the  Workload  Index  revealed  that 
only  13%  of  the  452  shifts  were  understaffed  by  -2  personnel,  several 
indirect  care  activities  were  judged  as  completed  inadequately  by  a  large 
percentage  of  respondents.  In  fact,  perceptions  of  administrative  duties 
completed  (x  =  2.9),  of  patient  rounds  made  with  physicians  (x  =  2.8),  and  of 
care  plans  written  (x  =  2.8)  were  less  than  adequate.  A  parallel  study 
conducted  by  Misener  and  Freline  (1983)  reported  that,  across  nine  Army 
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hospitals,  time  spent  by  nursing  care  providers  was  distributed  as  28.5%  for 
direct  care,  56.5%  for  indirect  care,  and  15%  as  unavailable  for  patient 
care.  In  addition,  a  re-examination  of  results  from  a  study  conducted  by 
Kelly  (1980)  in  three  naval  hospitals  provided  support  for  these  staff  time 
distributions.  The  time  percentages  were  further  validated  by  results 
reported  by  Lake  (1982)  for  the  civilian  community.  Although  the  operational 
definitions  and  categorizations  differed  slightly,  the  percentages  of  time 
spent  on  direct  and  indirect  care  within  the  military  and  civilian  setting 
did  not  differ  substantively  (Misener  &  Freline,  1983).  The  WMSN  system 
evaluated  in  this  study  did  not  include  unavailable  time  and  provided  45-75% 
indirect  care  time  for  various  hospitals.  Based  on  the  above,  a  requirements 
model  predicated  on  the  current  WMSN  percentages  does  not  appear  to  give 
adequate  time  for  indirect  care  functions. 

A  significant  finding  was  the  relationship  between  the  Workload  Index  and 
nurses'  perceptions  of  how  well  direct  and  indirect  nursing  care  was 
provided.  Although  classification  systems  have  been  judged  as  extremely 
effective  in  matching  workload  to  numbers  of  staff,  the  linkage  between 
quantity  of  nursing  personnel  to  quality  of  outcome  has  been  elusive.  The 
results  of  this  study  demonstrate  that  quality,  as  defined  by  perceptions  of 
how  "well"  nursing  care  was  provided,  is  linearly  related  to  the  numbers  of 
staff  available  to  give  that  care  on  a  nursing  unit.  Granted,  these  results 
do  not  specifically  address  patient  outcomes,  but  past  research  has  shown 
that  professional  nursing  judgment  accurately  reflects  the  "actual"  unit 
situation  (Giovannetti  &  Mayer,  1984).  Therefore,  these  results  give  some 
credence  to  the  assumptions  that  quantity  and  category  of  staff  (i.e.,  RN) 
are  directly  related  to  quality  of  care  provided. 


In  general,  nurses  were  satisfied  with  the  system,  perceiving  it  to  be 
useful  for  management  purposes.  However,  this  perception  of  satisfaction  was 
significantly  affected  by  whether  the  nurse  was  involved  with  the  entire 
system,  or  with  only  the  patient  classification  portion,  and  by  the  specialty 
unit  on  which  the  nurse  worked.  Except  for  two  factors,  charge  nurses'  and 
staff  nurses'  perceptions  of  how  "well"  care  was  provided  were  similar  for 
both  direct  and  indirect  patient  care  activities.  Charge  Nurses'  perceptions 
of  quality  of  care  given  and  staffing  adequacy  on  a  particular  shift  were 
significantly  related  to  staffing  levels. 

A  review  of  written  commentaries  from  the  study  sample  of  434  Registered 
Nurses  revealed  several  generalizations  that  could  be  made  across  hospitals. 
These  included: 

1.  Staff  nurses  were  confused  about  what  types  of  activities  are 
included  in  the  patient  classification  Critical  Indicator  sheet. 
Numerous  recommendations  were  received  to  include  items  that  were 
indirect  care  tasks. 

2.  The  most  frequently  suggested  changes  to  the  direct  care 
activities  were  to  include  time  for  discharge  teaching  and 
isolation,  and  more  time  for  providing  care  to  a  newborn. 

3.  Dissatisfaction  with  the  system  was  greater  among  nurses  who 
worked  in  specialty  units,  i.e..  Labor  and  Delivery,  Intensive 
Care,  and  Nursery. 

4.  Sources  of  frustration  with  the  system  focused  on  non-avail- 
ability  of  additional  manpower  to  cover  staffing  deficits  and 
perceptions  that  management  does  not  use  the  WMSN  as  a  staffing 


6.  CONCLUSIONS. 

As  a  result  of  this  study,  the  following  conclusions  are  made: 

1.  The  WMSN  patient  classification  tool  was  found  to  be  a  reliable 
tool  for  measuring  nursing  care  hour  requirements  by  patient 
category  across  all  study  hospitals.  Thus,  the  first  hypothesis 
was  supported. 

2.  The  WMSN  classification  tool  reliably  measured  nursing  care 
requirements  on  all  specialty  units  except  medicine.  Thus,  the 
second  hypothesis  was  partially  supported. 

3.  There  was  a  high  degree  of  reliability  among  most  factors  in  the 
patient  classification  instrument.  Of  the  ten  factors,  only 
complex  treatments,  teaching,  simple  treatments,  and  emotional 
support  resulted  in  an  ICC  value  below  .90.  Thus,  the  third 
hypothesis  was  supported. 

4.  The  validity  coefficient  for  the  classification  tool  was 
established  by  comparing  it  to  an  existing  tool  which  possessed 
both  content  and  criterion-related  validity.  The  results 
demonstrated  a  high  correlation  between  the  two  instruments,  thus 
supporting  the  fourth  hypothesis. 

5.  A  majority  (74.4%)  of  the  sample  expressed  satisfaction  or  a 
neutral  attitude  toward  the  system.  Perceptions  of  satisfaction 
were  highly  correlated  with  having  reviewed  or  having  had  input 
into  staffing  decisions.  Thus,  the  fifth  hypothesis  which 
specified  finding  greater  satisfaction  than  dissatisfaction 
with  the  system  was  supported. 
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6.  Charge  nurses'  perceptions  of  staffing  adequacy  and  quality  of 
care  given  were  significantly  (positively)  related  to  staffing  as 
recommended  by  the  WMSN.  Thus,  the  sixth  hypothesis  was 
supported. 

7.  Nurses'  perceptions  of  how  well  direct  care  and  indirect  care 
activities  were  provided  to  patients  were  significantly  related 
to  quantity  of  staff,  particularly  registered  nurse  staff,  as 
measured  by  the  WMSN  Workload  Index.  Thus,  the  seventh 
hypothesis  was  supported. 

7.  RECOMMENDATIONS 

This  section  is  divided  into  two  parts.  The  first  presents  recommenda¬ 
tions  that  have  already  been  implemented.  By  describing  these,  the  authors 
will  provide  documentation  of  the  transition  that  has  occurred  in  the  system 
since  the  completion  of  this  evaluative  study.  The  second  section  addresses 
those  recommendations  that  are  still  being  considered  or  are  ongoing  in 
execution. 

a.  Completed  Recommendations. 

1.  Based  on  the  study  findings,  changes  in  the  critical  indicator 
tool  to  incorporate  discharge  teaching  and  isolation  were  made. 
Point  values  on  some  critical  indicators  were  also  adjusted  based 
on  the  results  of  the  validity  study. 

2.  The  percentages  of  direct,  indirect,  and  non-available  time  by 
specialty  services  were  adjusted  to  more  accurately  reflect  staff 
time  distributions.  With  this  change  the  original  indirect  care 
time  breakdowns  (unpredicted  needs,  teaching  hospital  allowance. 
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and  semi-private  room  allowances)  were  eliminated.  Therefore, 
nursing  care  time  for  category  of  care  (Class  I-VI)  now  differs 
slightly  depending  on  the  type  of  patient  being  treated,  i.e., 
psychiatric,  medical-surgical,  obstetric  and  gynecologic, 
pediatric,  critical  care,  or  normal  newborn. 

3.  The  charge  nurse  and  sinior  corpsperson  assigned  to  a  unit  during 
the  week  were  eliminated  from  the  staff  counted  as  direct  care 
providers.  These  two  staff  members  provide  support  and  act  as 
resources  to  direct  care  staff.  The  administrative  duties  they 
perform  (staffing,  evaluations,  orientation  of  new  staff)  were 
excluded  from  the  indirect  care  percentages. 

4.  The  WMSN  Educational  Workbook  was  re-written  and  distributed  to 
all  hospitals  in  June  1985  to  disseminate  the  changes.  Along 
with  the  workbook,  new  Nursing  Care  Hour  Requirements  Charts  by 
specialty  unit,  with  their  corresponding  Personnel  Requirements 
Charts  for  both  8  hour  and  12  hour  shifts,  were  provided.  All 
hospitals  were  given  a  1  September  1985  implementation  date. 

5.  To  continue  development  of  the  system,  a  Nurse  Corps  officer  has 
been  assigned  to  the  Research  Department  at  NSHS  as  the  WMSN 
Project  Officer.  Position  duties  include  managing  the  system, 
refining  the  patient  classification  instrument  for  specialty 
areas,  and  assisting  in  or  conducting  studies  which  involve 
analysis  of  nursing  workload. 

b.  Proposed  Recommendations. 

1.  Because  the  current  WMSN  does  not  address  patient  care 

requirements  in  Labor  and  Delivery,  Recovery  Room,  or  Ambulatory 
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Care,  it  is  recommended  that  the  system  be  further  developed  to 
encompass  nursing  workload  in  these  areas. 

2.  Due  to  the  findings  that  staff  nurses  were  unclear  as  to  the 
composition  of  the  patient  classification  tool,  it  is  recommended 
that  orientation  to  the  system  be  ongoing  with  emphasis  on 
documented  proficiency  in  classifying  patients. 

3.  To  support  learning  and  to  standardize  teaching,  it  is 
recommended  that  a  video  tape  be  created  to  complement  the 
workbook  and  be  distributed  to  all  hospitals. 

4.  To  institutionalize  the  nursing  system,  it  is  recommended  that  a 
Naval  Medical  Command  instruction  be  written  and  that 
standardized  forms  be  made  available  through  the  Government 
Printing  Office. 

5.  It  is  recommended  that  reliability  in  use  of  the  system  be 
assessed  monthly  with  mandatory  retraining  if  percentage  of  rater 
agreement  falls  below  SO  percent. 

6.  Validity  assessment  of  both  the  classification  and  staffing 
portions  of  the  system  should  be  ongoing.  It  is  recommended  that 
the  critical  indicators  and  their  corresponding  weighted  time 
values  be  re-validated  every  two  years  to  keep  pace  with  changes 
in  technology  and  professional  practice.  The  percentages  of 
staff  time  spent  in  direct  and  indirect  care  activities  should 
also  be  re-validated  periodically  using  sampling  techniques. 

7.  To  aggregate  and  collate  facility  workload  information  in  a 
usable  format  and  timely  manner,  it  is  recommended  that  nursing 
services  process  their  data  on  microcomputers.  A  Zenith  software 
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program  written  specifically  for  generating  WMSN  output  reports 
is  presently  available. 

8.  To  minimize  data  entry  time  and  ensure  accurate  input  of  the 
patient  classification  information,  a  mark  sense  version  of  the 
Critical  Indicator  Sheet  is  currently  being  developed  in 
cooperation  with  the  Army.  Data  from  these  forms  could  be 
automatically  entered  into  a  microcomputer  using  an  optical 
scanner.  Therefore,  it  is  recommended  that  scanners  be  made 
available  to  all  nursing  services. 

9.  Once  a  mechanism  for  downloading  facility  specific  data  to  the 
Naval  Medical  Data  Services  Center  mainframe  has  been  developed, 
it  is  recommended  that  quarterly  reports  which  aggregate  nursing 
workload  data  across  facilities  be  produced  for  headquarters 
decision-making.  This  concurrent  picture  of  staffing  requirement 
based  upon  patient  needs  could  be  used  to  validate  the  CNO 
directed  Navy  Manpower  Engineering  Program  Staffing  Standards  for 
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APPENDIX  A 

NAVAL  HOSPITAL  CHERRY  POINT 
CHERRY  POINT,  NORTH  CAROLINA 


APPENDIX  A-1 


DEMOGRAPHIC  PROFILE  OF  NURSE  PARTICIPANTS 
AT  NAVAL  WSPITAL  CHERRY  POINT 


Sex: 

Frequency 

Percent 

Males 

5 

25t 

Females 

15 

75t 

n  *  20 

loot 

B.  Age  Range: 

40X  of  nurses  are  under  age  35  years 
60X  of  nurses  are  under  age  40  years 


C,  Nursing  position: 


Frequency  Percent 


Staff  Nurses  16 
Charge  Nurses  3 
Supervisors,  Administrators,  Others  1 


sot 

15t 

5% 


APPENDIX  A-2 


COMPARISON  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BETWEEN  NURSE  EXPERT  AND  CHARGE  NURSE  CLASSIFIERS 
USING  THE  CRITICAL  INDICATOR  INSTRUMENT 
AT  NAVAL  HOSPITAL  CHERRY  POINT 


NURSE  EXPERT 
RATING 


CHARGE  NURSE  RATING 


FREQUENCY! 

PERCENT 

^Category 

Category 

Category 

ROW  PCT 
COL  PCT 

1 

1 

1 

1 

2 

1 

3 

L 

1 

4 

1 

1 

1 

0 

1 

22.22 

1 

5.56 

1 

0.00 

Category 

1 

80.00 

i 

20«00 

1 

0.00 

1 

80.00 

1 

11. LI 

1 

0.00 

2 

1 

1 

1 

8 

1 

0 

! 

5.56 

» 

44.44 

0.00 

Category 

1 

ll.ll 

1 

88.89 

1 

0.00 

1 

20.00 

1 

88.89 

1 

0.00 

3 

1 

0 

f 

0 

1 

4 

! 

0.00 

\ 

0.00 

1 

22.22 

Category 

1 

0.00 

1 

0.00 

1 

100.00 

1 

0.00 

1 

0.00 

1 

100.00 

TOTAL 

5 

9 

4 

27.78 

50.00 

22.22 

TOTAL 

5 

27-.  78 


9 

50.00 


4 

22.22 


18 

100.00 


Percent  of  Agreement  »  16/18  or  88.89% 

Kappa  Statistic  -  .82 

Standard  Deviation  of  Kappa  =  .133 

Z  Score  =6.9 

Category  agreement  significant  at  p  .001  using  Kappa  Statistic. 
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APPENDIX  A-3 


INTER-RATER  RELIABILITIES  FOR  FACTORS  ON  THE  CRITICAL 
INDICATOR  INSTRUMENT  AS  ESTIMATED  BY  ICC  AT 
NAVAL  HOSPITAL  CHERRY  POINT 

Factor  Intra-Class  Correlation  (ICC) 

Vital  Signs  .983 

Monitoring  .956 

Activities  of  Daily  Living  .694 

Feeding  .987 

Sinple  Treatments  .394 

Complex  Treatments  — 

Respiratory  Therapy  .434 

Intravenous  Therapy  .847 

Teaching  .917 

Emotional  Support  -.150 

Continuous  Care  — 

Total  .921 

All  factors  were  statistically  significant  at  p  <  .0001  level  using 
the  F  test. 


APPENDIX  A-4 


PERCEPTIONS  OF  ACCURACY  OF  THE  WMSN  IN  REFLECTING  THE  LEVEL 
OF  CARE  GIVEN  BY  NURSING  POSITION 
AT  NAVAL  HOSPITAL  CHERRY  POINT 


Chi  Square  =  5.218 

Degrees  of  Freedom  =  2 

Probability  *  .0736 

Note:  5  cells  have  less  than  5  cases. 


APPENDIX  A-5 


NURSING 

POSITION 


PERCEPTIONS  OF  USEFULNESS  OF  THE  WMSN 
AS  A  MANAGEMENT  TOOL  BY  NURSING  POSITION 
AT  NAVAL  HOSPITAL  CHERRY  POINT 


USEFULNESS  OF  WMSN  AS  A  MANAGEMENT  TOOL 


FREQUENCY 

ROW  PCT 

COL  PCT 

1 

1 

1 

USEFUL 

UNDECIDED 

NOT 

USEFUL 

t 

1 

TOTAL 

STAFF  NURSE 

1 

2  i 

7 

1 

7 

1 

16 

1 

12.50  i 

43.75 

1 

43.75 

1 

1 

40.00  1 

8  7.50 

1 

100.00 

1 

CHARGE  NURSE 

m  ^ 

1 

2  t 

1 

1 

0 

1 

3 

1 

.67  1 

33.33 

1 

0.00 

1 

( 

40.00  i 

12.50 

1 

0.00 

J 

SUPERVISORY 

J 

1  1 

0 

1 

0 

J 

1 

1 

LOO. 00  1 

0.00 

1 

0.00 

1 

1 

20.00  1 

0.00 

1 

0.00 

1 

TOTAL 

.5 

8 

7 

20 

Chi  Square  =  7.573 

Degrees  of  Freedom  =  4 

Probability  =  .1085 

Note:  7  cells  have  less  than  5  cases. 


APPENDIX  A-6 


SATISFACTION  WITH  THE  WMSN  AS  A  WHOLE  BY  NURSING  POSITON 
AT  NAVAL  HOSPITAL  CHERRY  POINT 


SATISFACTION  WITH  WMSN 

NURSING  FREQUENCY  I 

POSITION  ROW  FCT  I 

COL  PCT  *  SATISFIED  NEUTRAL  DISSATISFIED* 


1 

>  Am 

k  Am 

1 

TOTAL 

STAFF  NURSE 

1 

1 

1 

10 

r 

1 

S 

1 

16 

1 

6.25 

1 

62.50 

31.25 

1 

1 

25.00 

1 

,  Am 

1 00.00 

1 

83.33 

1 

CHARGE  NURSE 

1 

2 

1 

0 

1 

1 

1 

3 

1 

66.67 

1 

0.  00 

1 

33.33 

1 

1 

-  ^ - 

50.00 

1 

k 

0.00 

16.67 

1 

* 

SUPERVISORY 

1 

I 

1 

0 

1 

0 

1 

1 

1 

100.00 

1 

0.00 

1 

0.00 

1 

1 

25.00 

1 

>  *  — 

0.  00 

1 

.Am 

0.00 

1 

>♦ 

TOTAL 

4 

13 

6 

20 

Chi  Square  =  10.799 

Degrees  of  Freedom  =  4 

Probability  =  .0289 

Note:  8  cells  have  less  than  5  cases. 
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APPENDIX  A-9 


NURSING  PERSONNEL  STAFFING  LEVELS  FOR  EACH  SHIFT  AS 
DETERMINED  BY  WORKLOAD  INDEX  CRITERIA  AT 
NAVAL  HOSPITAL  CHERRY  POINT 


RN  STAFF 

Workload  Index 


Shifts 


Frequency 


Percent 


*Less  than  recommended 
**Recoainended 

***Greater  than  reconmended 


22 

2 


24 


91.67X 

8.33X 

lOOX 


NRN  STAFF 

« 

Workload  Index 


Shifts 


Frequency 

Percent 

*Less  than  recomnended 

1 

4.17X 

♦♦Recommended 

21 

87. 5X 

♦♦♦Greater  than  recommended 

2 

8.33t 

24 

loot 

TOTAL  STAFF 

Workload  Index 

Shifts 

Frequency 

Percent 

♦Less  than  recommended 

1 

4.17t 

♦♦Reconmended 

17 

70.83t 

♦♦♦Greater  than  recommended 

6 

25.00t 

24 

loot 

Key  to  Levels: 

♦Less  than  recommended: 

Minus  2  persons 

or  more 

♦♦Reconmended : 

Minus  1  person 

to  plus  1  p 

♦♦♦Greater  than  reconmended: 

Plus  2  persons 

or  more 

APPENDIX  B-1 


DEMOGRAPHIC  PROFILE  OF  NURSE  PARTICIPANTS 
AT  NAVAL  HOSPITAL  LEMOORE 


.  Sex: 

Frequency 

Percent 

Males 

4 

31% 

Females 

9 

69% 

n  *  13 

100% 

B.  Age  Range: 

69%  of  nurses  are  under  age  35  years 
100%  of  nurses  are  under  age  40  years 


Nursing  position: 

Frequency 

Percent 

Staff  Nurses 

11 

84.6% 

Charge  Nurses 

2 

15.4% 

Supervisors,  Administrators,  Others 

... 

... 

APPENDIX  B>2 


COMPARISON  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BETWEEN  NURSE  EXPERT  AND  CHARGE  NURSE  CLASSIFIERS 
USING  CRITICAL  INDICATOR  INSTRUMENT 
AT  NAVAL  HOSPITAL  LEMOORE 


Nurse  Expert 

Racing  Charge  Nurse  Racing 


FREQUENCYl 
PERCENT  1 


ROW  PCT 

1 

Category 

Category 

Category 

COL  PCT 

1 

1  1 

2 

1 

3 

1 

TOTAL 

1 

1 

3  1 

1 

1 

0 

1 

4 

i 

16.67  1 

5.56 

1 

0.00 

1 

22.22 

Category 

1 

75.00  1 

2  5.00 

1 

0.00 

1 

1 

100.00  1  7.14 

1 

0.00 

1 

2 

1 

0  i  13 

( 

0 

1 

13 

1 

0.00  1 

72.22 

1 

0.00 

1 

72.22 

Category 

1 

0.00  1 

100.00 

1 

0.00 

1 

1 

0.00  1 

92.86 

1 

0.00 

1 

3 

1 

0  1 

0 

1 

1 

1 

I 

1 

0.00  1 

0.00 

1 

5.56 

1 

5.56 

1 

0.00  1 

0.00 

1 

100.00 

1 

Category 

1 

0.00  1 

0.00 

1 

100.00 

1 

■  ■  ■  ■ 

_  _  1  "1  ■  - - 

TOTAL 

3 

14 

1 

18 

16.67 

77.78 

5.56 

100.00 

Percent  of  Agreement  =  17/18  or  97.4435 
Kappa  Statistic  »  .86 
Standard  Deviation  of  Kappa  =  .1365 
Z  Score  »  6.3 

Category  agreement  was  significant  at  p  <  .001  using  Kappa  Statistic. 
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INTER-RATER  RELIABILITIES  FOR  FACTORS  ON  THE  CRITICAL 
INDICATOR  INSTRUMENT  AS  ESTIMATED  BY  ICC  AT 
NAVAL  HOSPITAL  LEMOORE 


Factor 
Vital  Signs 
Monitoring 

Activities  of  Daily  Living 
Feeding 

Sinple  Treatments 
CoflV)1ex  Treatments 
Respiratory  Therapy 
Intravenous  Therapy 
Teaching 

Emotional  Support 
Continuous  Care 

Total 


Intra-Class  Correlation  (ICC) 
.894 
.610 
.981 
.896 
.772 

.970 

.930 

.923 


.909 


All  factors  were  statistically  significant  at  p  4  .0001  level  using 
the  F  test. 
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PERCEPTIONS  OF  ACCURACY  OF  THE  WMSN  IN  REFLECTING  THE  LEVEL 
OF  CARE  GIVEN  BY  NURSING  POSITION  AT  NAVAL  HOSPITAL  LEMOORE 


NURSING 

POSITION 


FREQUENCY 

1 

PF.RCEPTIONS  OF  ACCURACY  OF  WMSN 

ROW  PCT 

f 

CQL  PCT 

1 

1 

USUALLY 

SOIETIMES 

1 

TOTAL 

STAFF  NURSE 

1 

6 

1 

4 

1 

10 

li 

60.00 

1 

40.00 

1 

I 

85.71 

1 

80.00 

1 

CHARGE  NURSE 

L 

1 

1 

1 

2 

ll 

50.00 

1 

50.00 

1 

• 

1 

14.29 

1 

1^1 

20.00 

1 

-  ■«- 

total 

7 

5 

12 

Missing  Cases  =  1 

Chi  Square  *  .069 

Degrees  of  Freedom  =  1 

Probability  *  .7934 

Note:  3  cells  have  less  than  5  cases. 
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NURSING 

POSITION 


PERCEPTIONS  OF  USEFULNESS  OF  THE  WMSN  AS  A  MANAGEMENT  TOOL 
BY  NURSING  POSITION  AT  NAVAL  HOSPITAL  LEMOORE 


USEFULNESS  OF  WMSN  AS  A  MANAGEMENT  TOOL 


FREQUENCY 
ROW  PCT 
COL  PCT 


STAFF  NURSE 


I 

I  USEFUL  UNDECIDED  j 

t  5  1  5  1 

1  50.00  I  50.00  1 

I  83.33  (  83.33  I 


CHARGE  NURSE  I  I  ' 

f  50.00  J 

fc  16.67  I 

TOTAL  ^ 


1  I 
50.00  I 
16.67  I 

6 


TOTAL 

10 


2 


12 


Missing  Cases  =  1 

Chi  Square  =  .000 

Degrees  of  Freedom  =  1 

Probability  =  1 

Note:  Cells  so  sparse  not  valid. 
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NURSING 

POSITION 


SATISFACTION  WITH  THE  WMSN  AS  A  WHOLE  BY  NURSING  POSITION 
AT  NAVAL  HOSPITAL  LEMOORE 


FREOUGNCY 
ROW  PCT 
COL  PCT 


STAFF  MURSE 


•  SATISFACTION  WITH  WMSN 

I 

1‘SATISPIZD  NEUTRAL  DISSATISFIEfl 

_  I 

4  1  4  t  II 

|!  44.44  I  44.44  I  11.  H  | 

I  80.00  t  80.00  I  100.00  I 


CHARGE  NURSE  f  I  I 

I  50.00  I 
I  20.00  I 


total  5 


I  i  0  1 

50.00  i  0.00  I 

20.00  I  0.00  I 


TOTAL 

9 


2 


11 


Chi  Square  =  .244 
Degrees  of  Freedom  *  2 
Probability  =  .8856 


APPENDIX  B-7 


RANK  ORDER  OF  MAJOR  STRENGTHS  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  LEMOORE 


Variable 


Agree 

Frequency  Percent 


Comprehensive 
Ease  Of  Use 

Usefulness  As  a  Management  Tool 
Takes  Little  Time  To  Do 
Accurately  Reflects  Workload 
Reliable 


6 

4 

4 

3 

3 

2 


46.154% 

30.769% 

30.769% 

23.077% 

23.077% 

15.385% 


n  =  13  nurses 

(More  than  one  response  could  be  selected) 


APPENDIX  B-8 


RANK  ORDER  OF  MAJOR  WEAKNESSES  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  LEMOORE 


Variable 


Agree 

Frequency  Percent 


Unreliable 

5 

38.462t 

Not  Conprehensive 

4 

30.769t 

Takes  Long  Time  To  Do 

3 

23.077X 

Difficult  To  Use 

2 

15.385X 

Inaccurate  In  Reflecting  Workload 

1 

7.692X 

n  =  13  nurses 

(More  than  one  response  could  be  selected) 


APPENDIX  B-9 


NURSING  PERSONNEL  STAFFING  LEVELS  FOR  EACH  SHIFT  AS 
DETERMINED  BY  WORKLOAD  INDEX  CRITERIA  AT 
NAVAL  HOSPITAL  LEMOORE 


A.  RN  STAFF 

Workload  Index 


Shifts 


Frequency 


*Less  than  recommended  — 
♦♦Recommended  25 
♦♦♦Greater  than  recomnended  _ 2 


Percent 

92.59t 

7.41X 


27  loot 


B.  NRN  STAFF 

Workload  Index 


Shifts 


♦Less  than  recommended 
♦♦Recomnended 

♦♦♦Greater  than  recomnended 


C.  TOTAL  STAFF 

Workload  Index 


♦Less  than  recomnended 
♦♦Recomnended  level 
♦♦♦Greater  than  recomnended 


Key  to  Levels: 

♦Less  than  recomnended: 
♦♦Recommended : 

♦♦♦Greater  than  reromnended: 


Frequency  Percent 


26  96. 3t 

27  loot 

Shifts 

Frequency  Percent 

23  85.19* 

4  14.81* 


27  100* 


Minus  2  persons  or  more 
Minus  1  person  to  plus  1  person 
Plus  2  persons  or  more 


B-IQ 


APPENDIX  C-1 


DEMOORAPHIC  PROFILE  OF  NURSE  PARTICIPANTS 

AT  NAVAL  HOSPITAL  CAMP  PENDLETON 

Sex: 

Frequency 

Percent 

Males 

8 

14X 

Females 

49 

aex 

n  =  57 

100% 

B.  Age  Range: 

56X  of  nurses  are  under  age  35  ^ars 
70X  of  nurses  are  under  age  40  ^ars 

C,  Nursing  position: 

Staff  Hurses 
Charge  Nurses 

Supervisors,  Administrators,  Others 


Frequency 


40 

7 

10 


Percent 


70. 2t 
12. 3X 
17. 5X 


APPENDIX  C-2 

COMPARISON  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BETWEEN  NURSE  EXPERT  AND  CHARGE  NURSE  CLASSIFIERS 
USING  THE  CRITICAL  INDICATOR  INSTRUMENT 
AT  NAVAL  HOSPITAL  CAMP  PENDLETON 


NURSING  EXPERT 
RATING 


CHARGE  NURSE  RATING 


FREQUENCY 
PERCENT 
ROi<  PCT 
COL  PCT 


Category 


Category 


Category 


TOTAL 


Category  Category  Category  Category 


1  I 


33.33  I 
89.47  I 
100.00  1 

0  i 
O.QQ  i 
0.00  I 
0.00  i 


2  I 

2  I 
3.92  i 
10.53  I 
13.33  I 

13  I 
25.49  I 
81.25  I 
86.67  I 


3  I 


0 

0.00  I 
0.00  1 
0.00  I 

3  I 
5.88  t 
18.75  I 
21.43  I 


4  I 


0.00  I 
0.00  I 
0.00  I 

0  I 
0.00  I 
0.00  I 
0.00  i 


3  1 

0 

1 

0 

i 

11  t 

1 

1 

1 

0.00 

\ 

0.00 

1 

21.57  i 

1.96 

1 

Category  | 

0.00 

1 

0.00 

1 

91.67  1 

8.33 

1 

1 

0.00 

L 

0.00 

i 

78.57  1 

20.00 

1 

0.00  I  0.00  I  0.00  I  7.84  I 

0.00  I  0.00  I  0.00  I  100.00  I 

0.00  1  0.00  I  0.00  i  80.00  I 

17  15  14  5 

33.33  29.41  27.45  9.80 


37.25 


16 

31.37 


7.84 


51 

100.00 


Percent  of  Agreement  =  45/51  or  88.24X 
Kappa  Statistic  =  .83 
Standard  Deviation  of  Kappa  =  .063 

Z  Score  =  13.2 

Category  agreement  was  significant  at  p  .001  using  Kappa  Statistic. 


APPENDIX  C-3 


INTER-RATER  RELIABILITIES  FOR  FACTORS  ON  THE  CRITICAL 
INDICATOR  INSTRUMENT  AS  ESTIMATED  BY  ICC  AT 
NAVAL  HOSPITAL  CAMP  PENDLETON 


Factor 
Vital  Signs 
Monitoring 

Activities  of  Daily  Living 
Feeding 

Simple  Treatments 
Complex  Treatments 
Respiratory  Therapy 
Intravenous  Therapy 
Teaching 

Emotional  Support 
Continuous  Care 

Total 


Intra-Class  Correlation  (ICC) 
.989 
.931 
.9035 
.965 
.881 
.793 
.904 
.873 
.717 
.649 


.953 


All  factors  were  statistically  significant  at  p  ^  .0001  level  using 
the  F  test. 


APPENDIX  C-4 


HORSING 

POSITION 


PERCEPTIONS  OF  ACCURACY  OF  THE  WMSN  IN  REFLECTING  THE  LEVEL 
OF  CARE  GIVEN  BY  NURSING  POSITION  AT 
NAVAL  HOSPIVAL  CAMP  PENDLETON 


PERCEPTIONS  OF  ACCURACY  OF 

UMSN 

FREQUENCY 

1 

ROW  PCT 

/ 

COL  PCT 

ifuSOALIT  SOMETIMES  NEVER 

1 

--  »  »- — — <  ---  '■ 

1.  TOTAL 

STAFF  NURSE 

11 

15 

1 

23 

1 

2 

1 

40 

1 

37.50 

1 

57.50 

1 

5.00 

1 

1 

60.00 

1 

82.14 

1 

100.00 

1 

CHARGE  NURSE 

|l 

5 

1 

1 

1 

0 

1 

6 

1 

63.33 

1 

16.67 

1 

0.00 

1 

i 

20.00 

1 

3.57 

1 

0.00 

1 

1  1  II  1  1 

SUP ERV  ISORY 

1 

5 

1 

4 

1 

0 

1 

9 

1 

55.56 

1 

44.44 

1 

0.00 

1 

J 

20.00 

1 

14.29 

1 

0.00 

1 

total 


25  28  2  55 


Missing  Cases  =  2 

Chi  Square  »  5.200 

Degrees  of  Freedom  »  4 

Probability  =  .2674 

Note:  5  cells  have  less  than  5  cases. 


APPENDIX  C-5 


NURSING 

POSITION 


PERCEPTIONS  OF  USEFULNESS  OF  THE  WMSN  AS  A  MANAGEMENT  TOOL 
BY  NURSING  POSITION  AT  NAVAL  HOSPITAL  CAMP  PENDLETON 


USEFULNESS  OF  WMSN  AS  A  MANAGEMENT  TOOL 


FREQUENCY 

1 

ROW  PCT 

1 

COL  PCT 

1 

NOT 

1 

USEFUL 

UNDECIDED 

USEFUL 

1 

total 

STAFF  NURSE 

1 

16  i 

12 

1  12 

1 

40 

1 

40.00  I 

30.00 

1  30.00 

1 

1 

53.33  1  85.71 

i  100.00 

1 

CHARGE  NURSE 

1 

5  J  1 

1  0 

1 

6 

1 

83.33  1 

16.67 

1  0.00 

1 

1 

16.67  I 

7.14 

i  0.00 

1 

SUP ERVISQRY 

1 

9  1 

1 

I  0 

1 

10 

1 

93.00  t 

10.00 

1  0.00 

1 

1 

30.00  1 

7.14 

1  0.00 

1 

TOTAL 

30 

14 

12 

♦ 

56 

Missing  Cases  =  1 

Ch1  Square  >  11.111 

Degrees  of  Freedom  =  4 

Probab 11 1 ty  =  . 0253 

Mote:  4  cells  have  less  than  5  cases. 


APPENDIX  C<6 


SATISFACTION  WITH  THE  WMSN  AS  A  WHOLE  BY  NURSING 
POSITION  AT  NAVAL  HOSPITAL  CAMP  PENDLETON 


SATISFACTION  WITH  WMSN 


NURSING 

POSITION 

FREQUENCY 

ROW  PCT 

COL  PCT 

1 

1 

t 

SATISFIED 

NEUTRAL 

DISSATISFIHp 

1 

TOTAL 

STAFF  NURSE 

1 

9 

w - 

1 

15 

1 

15  1 

39 

I 

23 .08 

I 

38.46 

1 

38.46  t 

1 

52.94 

1 

mi 

75.00 

1 

83.33  1 

CHARGE  NURSE 

1 

3 

1 

2 

1 

2  1 

7 

1' 

42.86 

1 

28.57 

1 

28.57  1 

17.65 

I 

10.00 

11. ll  1 

SUPERVISORY 

5 

1 

3 

r 

1 

!  1  ' 

9 

1 

55.56 

1 

33.33 

t 

ll.ll  1 

1 

29.41 

1 

15.00  I 

5.56  1 

TOTAL 

17 

20 

- 

18 

55 

Missing  Cases  *  2 

Chi  Sqaure  =  4.767 

Degrees  of  Freedom  =  4 

Probability  *  .3121 

Note:  5  cells  have  less  than  5  cases. 


APPENDIX  C-7 


RANK  ORDER  OF  MAJOR  STRENGTHS  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  CAMP  PENDLETON 


Variable 


Agree 

Frequency  Percent 


Ease  Of  Use 

19 

33.333X 

Usefulness  As  a  Management  Tool 

17 

29.825Jt 

Takes  Little  Time  To  Do 

14 

24.561t 

Reliable 

10 

17.544X 

Conprehensive 

8 

14.035X 

Accurately  Reflects  Workload 

8 

14.035X 

n  =  57  nurses 

(More  than  one  response  could  be  selected) 


APPENDIX  C-8 


RANK  ORDER  OF  MAJOR  WEAKNESSES  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  CAMP  PENDLETON 


Agree 

Variable  Frequency  Percent 


Inaccurate  In  Reflecting  Workload 

30 

52.632S 

Not  Coiiv)rehensive 

20 

35.088X 

Unreliable 

20 

35.088X 

Takes  Long  Time  To  Do 

13 

23.807X 

Not  Useful  As  A  Management  Tool 

5 

8.772X 

Difficult  To  Use 

3 

5.263X 

n  =  57  nurses 

(More  than  one  response  could  be  selected) 


APPENDIX  C-9 


NURSING  PERSONNEL  STAFFING  LEVELS  FOR  EACH  SHIFT  AS 
DETERMINED  BY  WORKLOAD  INDEX  CRITERIA  AT 
NAVAL  HOSPITAL  CAMP  PENDLETON 


RN  STAFF 

Workload  Index 


Shifts 


*Less  than  recomnended 
♦♦Recomnended 

♦♦♦Greater  than  recomnended 


NRN  STAFF 

Workload  Index 


♦Less  than  recommended 
♦♦Recomnended 

♦♦♦Greater  than  recomnended 


TOTAL  STAFF 

Workload  Index 


♦Less  than  recommended  level 
♦♦At  recomnended  level 
♦♦♦Greater  than  recommended  level 


Key  to  Levels: 

♦Less  than  recommended: 
♦♦Recomnended : 

♦♦♦Greater  than  recommended: 


Frequency 

Percent 

3 

4.76X 

59 

93.65X 

1 

1.591 

63 

1001 

Shifts 

Frequency 

Percent 

10 

15.871 

45 

71.431 

8 

12.71 

63 

1001 

Shifts 

Frequency 

Percent 

13 

20.641 

43 

68.251 

7 

11.111 

63 

1001 

Minus  2  persons  or  more 
Minus  1  person  to  plus  1  person 
Plus  2  persons  or  more 


APPENDIX  D-1 


DEMOGRAPHIC  PROFILE  OF  NURSE  PARTICIPANTS 
AT  NAVAL  HOSPITAL  CHARLESTON 


A.  Sex;  Frequency 

Males  15 

Females  49 

n  =  64 


B.  Age  Range: 

70t  of  nurses  were  under  age  35  years 
81X  of  nurses  were  under  age  40  years 


C,  Nursing  position:  Frequency 

Staff  Nurses  51 

Charge  Nurses  7 

Supervisors,  Administrators,  Others  6 

■  n  »  64 


Percent 

23. 4X 
76. 6X 

lOOX 


Percent 

79. 7X 
10. 9X 

9.4S 

100% 


APPENDIX  D-2 


COMPARISON  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BETWEEN  NURSE  EXPERT  AND  CHARGE  NURSE  CLASSIFIERS 
USING  THE  CRITICAL  INDICATOR  INSTRUMENT 
AT  NAVAL  HOSPITAL  CHARLESTON 

NURSING  EXPERT 

rating  charge  nurse  rating 


FREQUENCY 
PERCENT 
ROrf  PCT 
COL  PCT 


Category 


Category 


Category 


Category 


Category 


TOTAL 


Category  Category  Category  Category  Category 


5 

L3.51 

83.33 

83.33 

I 

2.70 
(>  .67 
16.67 

0 

0.00 

0.00 

0.00 


I  I  I 

I  2.70  I 

1  16.67  I 

I  6.67  I 


13  I  11 

35.14  I  2.70  I 
86.67  I  6.67  1 
86.67  I  7.69  I 

II  12  I 
2.70  I  32.43  I 
7.69  i  92.31  I 
6.67  I  92.31  I 


0  I 
0.00  I 
0.00  I 
0.00  I 

0  I 
0.00  I 
0.00  I 
0.00  I 

6 

16.22 


0  t 
0.00  I 
0.00  I 
0.00  I 

0  I 
0.00  I 
0.00  I 
0.00  I 

15 

40.54 


0  1  0  1  0  i 

0.00  i  0.00  I  0.00  I 

0.00  I  0.00  I  0.00  I 

0.00  I  0.00  I  0.00  I 

II.  01  01 

2.70  I  0.00  I  0.00  I 

6.67  1  0.00  I  0.00  1 

7.69  I  0.00  I  0.00  I 

12  I  0  1  0  1 

32.43  I  0.00  I  0.00  I 

92.31  I  0.00  I  0.00  I 

92.31  I  0.00  I  0.00  I 

0  1  2  1  0  1 

0.00  I  5.41  I  0.00  I 

0.00  I  100.00  I  0.00  I 

0.00  I  100.00  I  0.00  I 

0  1  0  1  11 

0.00  1  0.00  I  2.70  i 

0.00  I  0.00  I  100.00  I 

0.00  I  0.00  I  100.00  I 

13  2  1 

35.14  5.41  2.70 


TOTAL 

6 

16.22 


15 

40.54 


13 

35.14 


2 

5.41 


1 

2.70 


37 

100.00 


Percent  of  Agreement  =  33/37  or  89.22 
Kappa  Statistic  »  .84 

Standard  Deviation  of  Kappa  =  .0748 
Z  Score  =  11.247 

Category  agreement  was  significant  at  p  4  .001  using  Kappa  Statistic. 


D-3 


APPENDIX  0-3 


INTER-RATER  RELIABILITIES  FOR  FACTORS  ON  THE  CRITICAL 
INDICATOR  INSTRUMENT  AS  ESTIMATED  BY  ICC  AT 
NAVAL  HOSPITAL  CHARLESTON 


Factor 
Vital  Signs 
Monitoring 

Activities  of  Daily  Living 
Feeding 

Sinple  Treatments 
Co(M)1ex  Treatments 
Respiratory  Therapy 
Intravenous  Therapy 
Teaching 

Emotional  Support 
Continuous  Care 

Total 


Intra-Class  Correlation  (ICC) 
.928 
.996 
.997 
.959 
.918 
.828 
.976 
.976 
.793 
.763 


.991 


All  factors  were  statistically  significant  at  p  <  .0001  level  using 
the  F  test. 


APPENDIX  D-4 


PERCEPTIONS  OF  ACCURACY  OF  THE  WMSN  IN  REFLECTING 
THE  LEVEL  OF  CARE  GIVEN  BY  NURSING  POSITION 
AT  NAVAL  HOSPITAL  CHARLESTON 


NURSING 

POSITION 


FREQUENCY 
ROW  PCT 
COL  PCT 


STAFF  NURSE 


CHARGE  NURSE 


SUPERVISORY 


PERCEPTIONS  OF  ACCURACY  OF  WMSN 
^ALLT  SGKETIMES  NEVER  < 


I  TOTAL 


26  ( 


19  i 


52.00  I  38.00  I 


5  i 

10.00  I 


70.27  I  90.48  I  100.00  I 


5  1  2  1 

71.43  I  28.57  I 
13.51  I  9.52  I 


6  I 

100.00  I 

16.22  I 


0  I 
0.00  I 
0.00  I 


0  I 
0.00  I 
0.00  i 
. . 

0  i 
0.00  I 
0.00  I 


TOTAL 


Missing  Cases  *  I 

Chi  Square  *  5.992 

Degrees  of  Freedom  »  4 

Probability  »  .199 

Note:  4  cells  have  less  than  5  cases. 


APPENDIX  0-5 


PERCEPTIONS  OF  USEFULNESS  OF  THE  WMSN  AS  A  MANAGEMENT  TOOL 
BY  NURSING  POSITION  AT  NAVAL  HOSPITAL  CHARLESTON 


NURSING 

FREQUENCY 

1  USEFULNESS 

OF  WMSN 

AS 

A  MANAGEMENT  TOOL 

POSITION 

ROW  per 

f  1 

NOT 

COL  PCT 

ifeSEFUL 

UNDECIDED 

USEFUL  1 

1 

1 

TOTAL 

STAFF  NURSE 

1  24 

1 

19 

1 

7  1 

50 

1  46.00 

1 

38.00 

1 

14.00  ( 

1  68.57 

•  •  •  • 

1 

* 

90.48 

1 

100.00  ( 

CHARGE  NURSE 

1  5 

1 

2 

1 

0  1 

7 

1  71.43 

1 

28.57 

1 

0.00  1 

• 

1  14.29 

1 

9.52 

1 

0.00  1 

.4  .-....•..MM 

------- 

SUPERVISORY 

1  6 

1 

0 

1 

0  i 

6 

|!  100.00 

1 

0.00 

1 

0.00  1 

1  17.14 

1 

0.00 

1 

,  Am 

0.00  1 

TOTAL 

35 

21 

7 

63 

Missing  Cases  *  1 

Chi  Square  =  2.159 

Degrees  of  Freedom  »  4 

Probability  =  .1277 

Note:  4  cells  have  less  than  5  cases. 


APPENDIX  D-6 


SATISFACTION  WITH  THE  HMSN  AS  A  WHOLE  BY  NURSING  POSITION 
AT  NAVAL  HOSPITAL  CHARLESTON 


NURSING  FR60UGNCY 

POSITION  •^0*' 

COL  PCT 


STAFF  NURSE 

1 

16 

1 

18 

1 

16 

1 

1 

32.00 

1 

3  6.00 

1 

32.00 

1 

1 

^  t_ 

61 .54 

1 

90.00 

1 

94.12 

1 

■  ar  T  m 

CHARGE  NURSE 

1 

4 

1 

2 

1 

1 

1 

1 

57.14 

1 

28.57 

1 

14.29 

1 

1 

15.38 

1 

10.00 

1 

5.88 

{ 

SUPERVISORY 

1 

6 

1 

0 

1 

0 

1 

1 

100.00 

1 

0.00 

1 

0.00 

1 

1 

23.08 

1 

0.00 

1 

0.00 

1 

TOTAL 

26 

20 

17 

I  SATISFACTION  WITH  WMSN 

I 

\  SATISFIED  NEUTRAL  DISSATISFIED  j 


Missing  Cases  -  1 

Chi  Square  =  11.199 

Degrees  of  Freedom  =  4 

Probability  »  .0248 

Note:  5  cells  have  less  than  5  cases. 


APPENDIX  0-7 


RANK  ORDER  OF  MAJOR  STRENGTHS  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  CHARLESTON 


Variable 


Agree 

Frequency  Percent 


Usefulness  As  a  Management  Tool 

35 

54.688X 

Ease  Of  Use 

18 

28.12SX 

Takes  Little  Time  To  Do 

17 

26.563t 

Cooprehensive 

16 

25.000« 

Accurately  Reflects  Workload 

11 

17.188X 

Reliable 

8 

12.500X 

n  »  64  nurses 

(M  ^e  than  one  response  could  be  selected) 


APPENDIX  D-8 


RANK  ORDER  OF  MAJOR  WEAKNESSES  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  CHARLESTON 


Variable 


Agree 

Frequency  Percent 


Inaccurate  In  Reflecting  Workload 

27 

42.188X 

Not  Comprehensive 

17 

26.563t 

Unreliable 

13 

20.313t 

Takes  Long  Time  To  Do 

12 

18.75X 

Not  Useful  As  A  Managenent  Tool 

4 

6.2SX 

Difficult  To  Use 

1 

1.563t 

n  »  64  nurses 

(More  than  one  response  could  be  selected) 


APPENDIX  D-9 


NURSING  PERSONNEL  STAFFING  LEVELS  FOR  EACH  SHIFT  AS 
DETERMINED  BY  WORKLOAD  INDEX  CRITERIA  AT 
NAVAL  HOSPITAL  CHARLESTON 


A.  RN  STAFF 

Workload  Index  Shifts 


Frequency 

Percent 

*Less  than  recomnended 

4 

6.35X 

**Recomtnended 

56 

88.89X 

***Greater  than  recommended 

3 

4.76X 

63 

loot 

B.  NRN  STAFF 


Workload  Index  Shifts 


Frequency 

Percent 

*Less  than  recommended 

3 

4.76X 

♦♦Recommended 

59 

93.65X 

♦♦♦Greater  than  recommended 

1 

1.59X 

63 

lOOX 

C.  TOTAL  STAFF 


Workload  Index  Shifts 


Frequency 

Percent 

♦Less  than  recommended 

8 

12. 7« 

♦♦Recomnended  level 

47 

74. 6X 

♦^♦Greater  than  recommended 

-S 

12. 7X 

63 

lOOX 

Key  to  Levels: 

♦Less  than  recommended: 

Minus  2  persons 

or  more 

♦♦Recomnended : 

Minus  1  person 

to  plus  1  person 

♦♦♦Greater  than  recomnended: 

Plus  2  persons 

or  more 
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APPENDIX  E-1 


ft 


DEMOGRAPHIC  PROFILE  OF  NURSE  PARTICIPANTS 
AT  OAKLAND  NAVAL  HOSPITAL 


i 

• . 

L- 

S>-' 

I 


R 

K- 


A.  Sex: 


Males 

Females 


Frequency 

24 

79 

103 


B.  Age  Range: 

78X  of  nurses  are  under  age  35  ^ars 
92X  of  nurses  are  under  40  years 


C,  Nursing  position:  Frequency 

Staff  Nurses  78 

Charge  Nurses  10 

Supervisors,  Administrators,  Others  15 

n  =  103 


I 

r.' 

t. 

r. 

■ 


■ 


E-2 


'-■'.V.'  ■ 


_ — *  — ^  ^  ^  aA-  ^  ^  .  JK  .A  .  .A.  *— *  *'..A  ■>■  -g  '■ 


Percent 

23.3t 

76.7% 

100% 


Percent 

75.7% 

9.7% 

14.6% 

100% 


APPENDIX  E-2 


COMPARISON  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BET>iEEN  NURSE  EXPERT  ANO  CHARGE  NURSE  CLASSIFIERS 
USING  THE  CRITICAL  INDICATOR  INSTRUMENT 
AT  NAVAL  HOSPITAL  OAKLAND 


NURSE  EXPERT 
RATING 


charge  nurse  rating 


FREQUENCY  1 

PERCENT  i 

RON  PCT  1 

Category 

Category 

Category 

Category 

COL  PCT  1 

1 

1 

2 

1 

3 

I 

4 

1 

TOTAL 

1  1 

6 

1 

1 

1 

0 

1 

0 

1 

7 

1 

15.79 

1 

2.63 

1 

0.00 

1 

0.00 

1 

18.42 

Category  • 

85.71 

1 

14.29 

1 

0.00 

1 

0.00 

1 

85.71 

1 

6.67 

1 

0.00 

1 

0.00 

1 

2  1 

1 

1 

11 

1 

3 

1 

0 

1 

15 

1 

2.63 

1 

28.95 

1 

7.89 

1 

0.00 

1 

39.47 

Category  | 

6 .67 

1 

7S.33 

1 

20.00 

1 

0.00 

I 

\ 

14.29 

1 

T3.33 

1 

23.08 

1 

0.00 

1 

3  1 

0 

1 

3 

1 

10 

1 

0 

1 

13 

1 

0.00 

1 

7.89 

1 

26.32 

1 

0.00 

1 

34.21 

Category  1 

0.00 

1 

23.08 

1 

76.92 

1 

0.00 

1 

1 

0.00 

1 

20.00 

1 

76.  92 

(  0.00 

- - - - - 

1 

*■  * 

4  1 

0 

1 

0 

1 

1 

O  1 

1 

1 

3 

1 

3 

\ 

0.00 

i 

0.00 

1 

0.90 

1 

7.89 

1 

7.89 

Category  j 

0.00 

1 

0.00 

1 

0.00 

1 

100.00 

1 

1 

0.00 

1 

0.00 

1 

0.00 

1 

.  — 

100.00 

1 

TOTAL 

7 

15 

13 

3 

38 

18.42 

39.47 

34.21 

7.89 

100.00 

Percent  of  Agreement  =  30/38  or  78.95X 
Kappa  Statistic  =  .69 

Standard  Deviation  of  Kappa  =  .096 

Z  Score  *  7.2 

Category  agreement  was  significant  at  .001  using  Kappa  Statistic. 
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APPENDIX  E-3 


INTER-RATER  RELIABILITIES  FOR  FACTORS  ON  THE  CRITICAL 
INDICATOR  INSTRUMENT  AS  ESTIMATED  BY  ICC  AT 
NAVAL  HOSPITAL  OAJCUND 


Factor 

Vital  Signs 
Monitoring 

Activities  of  Daily  Living 
Feeding 

Single  Treatments 
Complex  Treatments 
Respiratory  Therapy 
Intravenous  Therapy 
Teaching 

Emotional  Support 
Continuous  Care 

Total 


Intra-Class  Correlation  (ICC) 

.991 

.996 

.894 

.915 

.546 

.748 

.961 

.981 

.862 

.457 


.963 


All  Factors  were  statistically  significant  at  p  4  .0001  level  using 
the  F  test. 


APPENDIX  E-4 


NURSING 

POSITION 


PERCEPTIONS  OF  ACCURACY  OF  THE  WMSN  IN  REFLECTING  THE  LEVEL 
OF  CARE  GIVEN  BY  NURSING  POSITION 
AT  NAVAL  HOSPITAL  OAKLAND 


PERCEPTIONS  OF  ACCURACY  OF  WMSN 

FREQUENCY  I 

ROW  PCT  I 


COL  PCT 

1 

USUALLY 

SOMETIMES 

NEVER 

1 

1 

1 

TOTAL 

STAFF  NURSE 

1 

32 

1 

40 

1 

3 

1 

75 

1 

1  42.67 

1 

53.33 

1 

4.00 

1 

li 

—  A 

64.00 

1 

88.89 

1 

75.00 

1 

*  ^ 

CHARGE  NURSE 

II 

1  6 

1 

3 

1 

1 

1 

10 

1 

60.00 

1 

30.00 

1 

10.00 

1 

1 

12.00 

1 

6.67 

1 

* - 

25.00 

1 

SUPERVISORY 

1 

12 

*  ^ 

1 

2 

1 

0 

*  ▼ 

1 

14 

II 

1  85.71 

1 

14.29 

1 

0.  00 

1 

J 

24.00 

1 

4.44 

1 

0.00 

1 

TOTAL 

SO 

, 

45 

4 

99 

Missing  *  4 
Chi  Square  »  10.514 
Degrees  of  Freedom  »  4 
Probability  =  .03 

Note:  5  cells  have  less  than  5  cases. 


APPENDIX  E-5 


PERCEPTIONS  OF  USEFULNESS  OF  THE  WMSN  AS  A  MANAGEMENT  TOOL 
BY  NURSING  POSITION  AT  OAKLAND  NAVAL  HOSPITAL 


NURSING 

POSITION 


FREQUENCY 
RQU  PCT 
COL  PCT 


STAFF  NURSE 


CHARGE  NURSE 


SUPERVISORY 


USEFULNESS  OF  WMSN  AS  A  MANAGEMENT  TOOL 


ijjSEFUL 


NOT 

UNDECIDED  USEFUL 


I  31  I  22  I  22  I 

|.  41.33  I  29.33  I  29.33  I 

I  62.00  I  84.62  I  91.67  i 


I  7  I 

I  70.00  I 
I  14.00  I 


I  12  I  21 

I  80.00  t  13.33  I 
I  24.00  I  7.69  I 


I  TOTAL 


22  I 


20.00  I  10.00  I 

7.69  i  4.17  I 

— — —  »  —  ■i-»  4. 

2  1  II 

13.33  I  6.67  I 

7.69  I  4.17  I 


total 


Missing  Cases  =  3 

Chi  Square  »  9.595 

Degrees  of  Freedom  =  4 

Note:  4  cells  have  less  than  5  cases. 
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APPENDIX  E^ 


NURSING 

POSITION 


SATISFACTION  WITH  THE  HMSN  AS  A  WHOLE  BY  NURSING 
POSITION  AT  NAVAL  HOSPITAL  OAKLAND 


SATISFACTION  WITH  WHSN 

FREQUENCY  | 

ROW  PCT  I  _ 

COL  PCT  |$ATIS7I£D  NEUTRAL  DISSATISFIED] 


I  I  TOTAL 


STAFF  NURSE 

1 

22 

1 

29 

1 

22 

«♦ 

1 

73 

1 

30.14 

1 

39.73 

1 

30.14 

1 

t 

-4I 

61.11 

1 

85.29 

1 

81.48 

1 

CHARGE  NURSE 

1 

4 

1 

3 

1 

3 

1 

10 

40.00 

1 

30.00 

1 

30.00 

1 

Il.lt 

1 

8.82 

1 

11.11 

1 

SUPERVISORY 

|l 

to 

1 

2 

1 

2 

1 

14 

1  1 

71.43 

1 

14.29 

1 

14.29 

1 

L 

27.78 

1 

5.88 

7.41 

1 

TOTAL 

36 

34 

27 

97 

Missing  Cases  *  6 

Chi  Square  *  8.751 

Degrees  of  Freedom  *  4 

Probability  ■  .0676 

Note:  5  cells  have  less  than  5  cases. 


APPENDIX  E-7 


RANK  ORDER  OF  MAJOR  STRENGTHS  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  OAKLAND 


Variable 


Agree 

Frequency  Percent 


Usefulness  As  a  Managesnent  Tool 

43 

41.748X 

Ease  Of  Use 

27 

26.214X 

Reliable 

20 

19.417t 

Takes  Little  Time  To  Do 

19 

18.447t 

Conv  rehens 1ve 

10 

9..709X 

Accurately  Reflects  Workload 

10 

9.709X 

n  =  103  nurses 

(More  than  one  response  could  be  selected) 


E-8 


APPENDIX  E>8 


RANK  ORDER  OF  MAJOR  WEAKNESSES  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  OAKLAND 


Agree 

Variable.  Frequency  Percent 


Not  Comprehensive 

41 

39.806X 

Inaccurate  In  Reflecting  Workload 

32 

32.06dS 

Unreliable 

29 

28.155X 

Takes  Long  Time  To  Do 

23 

22.33« 

Not  Useful  As  A  Management  Tool 

11 

10.68X 

Difficult  To  Use 

4 

3.883X 

n  =  103  nurses 

(More  than  one  response  could  be  selected) 
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APPENDIX  E-9 


NURSING  PERSONNEL  STAFFING  LEVELS  FOR  EACH  SHIFT  AS 
DETERMINED  BY  WORKLOAD  INDEX  CRITERIA  AT 
NAVAL  HOSPITAL  OAKLAND 


RN  STAFF 

workload  Index 


Shifts 


*Less  than  recomnended 
**Recoflinended 

♦♦♦Greater  than  recoinmended 


Frequency  Percent 


141  92.16X 

12  7.84t 

1S3  lOOX 


NRN  STAFF 


Workload  Index 


Shifts 


Frequency  Percent 


♦Less  than  recomnended 
♦♦Recommended 

♦♦♦Greater  than  recomnended 


7  4.58X 

139  90.85X 

7  4.58X 


153  loot 


TOTAL  STAFF 


Workload  Index 


Shifts 


Frequency  Percent 

♦Less  than  recomnended  13  8.50t 

♦♦Recommended  112  72.20t 

♦♦♦Greater  than  recomnended  28  18.30t 

153  loot 


Key  to  Levels: 

♦Less  than  recomnended; 
♦♦Recommended : 

♦♦♦Greater  than  recommended: 


Minus  2  persons  or  more 
Minus  1  person  to  plus  1  person 
Plus  2  persons  or  more 


APPENDIX  F-1 


DEMOGRAPHIC  PROFILE  OF  NURSE  PARTICIPANTS 
AT  NAVAL  HOSPITAL  PORTSMOUTH 


A.  Sex: 

Frequency 

Percent 

Males 

Females 

22 

155 

12. 4X 
87. 6X 

n  * 

177 

lOOX 

B.  Age  Range: 

61X  of  nurses  are  under  age  35  years 

SIX  of  nurses  are  under  40  years 

C,  Nursing  position: 

Frequency 

Percent 

Staff  Nurses 

Charge  Nurses 

Supervisors,  Administrators,  Others 

145 

24 

8 

81. 9X 
13. 6X 
4.5% 

n  = 

177 

lOOX 
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APPENDIX  F-2 


COMPARIi)ON  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BETWEEN  NURSE  EXPERT  AND  CHARGE  NURSE  CLASSIFIERS 
USING  THE  CRITICAL  INDICATOR  INSTRUMENT 
AT  NAVAL  HOSPITAL  PORTSMOUTH 


NURSE  EXPERT 
RATING 


CHARGE  NURSE  RATING 


FREQUENCY 


PERCENT  i 
ROW  pcr  1 

Category 

Category 

Category 

Category 

Category 

COL  PCT  I 

1 

1 

2 

1 

3 

1 

4  i 

5 

1 

TOTAL 

1  I 

7 

1 

3 

1 

0 

1 

0  1 

0 

1 

10 

1 

ia.4s 

1 

4.48 

1 

0.00 

1 

0.00  1 

0.00 

1 

14.93 

Category  1 

70.00 

1 

30.00 

1 

0.00 

1 

0.00  1 

0.00 

i 

1 

100.00 

1 

10.00 

0.  00 

1 

0.00  1 

0.00 

1 

2  1 

0 

1 

27 

1 

8 

1 

0  1 

0 

1 

35 

1 

0.00 

1 

40.30 

1 

11.94 

1 

0.00  1 

0.00 

1 

52.24 

Category  1 

0.00 

1 

77.14 

1 

22.86 

1 

0.00  1 

0.00 

1 

1 

0.00 

40  .oa 

38.10 

1 

0.00  i 

0.00 

1 

■  III  ■  ■  «  111!  _  ^ 

3  1 

0 

1 

0 

1 

13 

1 

1  1 

0 

1 

14 

I 

0.00 

1 

0.00 

1 

19.40 

1 

1.49  1 

O.Or 

1 

20.90 

^  ( 

0.00 

i 

0.00 

1 

92.86 

1 

7.14  1 

o.r> 

1 

C^Ccfory  1 

0.00 

1 

0.00 

1 

61.90 

1 

25.00  i 

0  .JO 

1 

4  1 

0 

1 

0 

1 

0 

1 

3  1 

1 

1 

4 

1 

o^oo 

1 

0.00 

1 

o.od 

1 

4.48  1 

1.49 

1 

5.97 

Category  1 

0.00 

1 

0.  00 

1 

0.00 

1 

75.00  1 

25.00 

1 

1 

0.00 

1 

0.00 

1 

0.00 

1 

75.00  1 

o 

• 

o 

o 

1 

■  ■  ■  1 

5  1 

0 

1 

0 

1 

0 

1 

0  1 

4 

1 

4 

1 

0.00 

1 

0.00 

1 

0.00 

1 

0.00  1 

5.97 

1 

5.97 

Category  I 

0.00 

1 

0.00 

f 

0.00 

1 

0.00  1 

100.00 

1 

0.00 

1 

0.00 

1 

0.00 

1 

0.00  1 

80.00 

1 

total 

7 

30 

21 

4  . 

5 

67 

'10.45 

44.78 

31.34 

5.97 

7.46 

100.00 

Percent  of  Agreement  =  54/67  or  80. 6X 
Kappa  Statistic  >  .713 
Standard  Deviation  of  Kappa  »  .071 

Z  Score  =  9.996 

Category  agreement  was  significant  at  p  .001  using  Kappa  Statistic. 
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APPENDIX  F-3 


INTER-RATER  RELIABILITIES  FOR  FACTORS  ON  THE  CRITICAL 
INDICATOR  INSTRUMENT  AS  ESTIMATED  BY  ICC  AT 
NAVAL  HOSPITAL  PORTSMOUTH 


Factor 
Vital  Signs 
Monitoring 

Activities  of  Daily  Living 
Feeding 

Single  Treatments 
Complex  Treatments 
Respiratory  Therapy 
Intravenous  Therapy 
Teaching 

Emotional  Support 
Continuous  Care 

Total 


Intra-Class  Correlation  (ICC) 
.917 
.945 
.971 
.926 
.639 

.967 

.921 

.414 

.248 


.905 


All  factors  were  statistically  significant  at  p  <  .0001  level  using 
tne  F  test. 
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APPENDIX  F-4 


PERCEPTIONS  OF  ACCURACY  OF  THE  WMSN  IN  REFLECTING  THE  LEVEL 
OF  CARE  GIVEN  BY  NURSING  POSITION 
AT  NAVAL  HOSPITAL  PORTSMOUTH 


PERCEPTIONS  OF  ACCURACY  OF  «MSN 


NURSING  FREQUENCY  I 

POSITION  ROW  PC  T  I 

COL  PCT  I  USUALLY  SOMETIMES  I 

I  I  TOTAL 


STAFF  NURSE  1 

65  1 

78 

1 

143 

1 

45.45 

1 

54.55 

1 

1 

73.03 

1 

92.86 

I 

:harge  nurse 

II 

18 

1 

5 

1 

23 

11 

78.26 

1 

21.74 

1 

1 

20.22 

1 

5.95 

1 

SUPERVISORY  1 

6 

1 

1 

1 

7 

1 

85.71 

1 

14.29 

1 

1 

6.74 

I 

1.19 

1 

TOTAL 

89 

84 

173 

Missing  Cases  =  4 

Chi  Square  ■  11.967 

Degrees  of  Freedom  =  2 

Probability  *  .0025 

Note:  1  cell  has  less  than  5  cases. 


APPENDIX  F-5 


NURSING 

POSITION 


PERCEPTIONS  OF  USEFULNESS  OF  THE  WMSN  AS  A  MANAGEMENT  TOOL 
BY  NURSING  POSITION  AT  NAVAL  HOSPITAL  PORTSIJIOUTH 


USEFULNESS  OF  UMSN  AS  A  MANAGEMENT  TOOL 


FREQUENCY 

1 

ROW  PCT 

1 

- 

COL  PCT  |i 

lySEFUL 

UNDECIDED 

NOT 

USEFUL 

1 

1 

TOTAL 

STAFF  NURSE 

1  55 

1 

59 

▼ 

1 

26 

1 

140 

I  39.29 

1 

42.14 

1 

18.57 

1 

1  67.90 

1 

93.65 

1 

96.30 

1 

CHARGE  NURSE 

1  19 

1 

3 

1 

1 

1 

23 

(  82.61 

1 

13.04 

1 

4.35 

1 

i  23.46 

1 

4.  76 

1 

3.70 

1 

SUPERVISORY 

1  7 

1 

1 

1 

0 

1 

8 

1  87.50 

1 

12.50 

1 

0.00 

1 

1  3.64 

1 

1.59 

1 

0.00 

1 

♦ 

TOTAL  81  63  27  171 


Missing  Cases  6 

C?ii  Square  *  20.427 

Degrees  of  Freedom  =  4 

Probabi 1 i ty  =  . 0004 

Note:  4  cells  haveless  than  5  cases. 


APPENDIX  F-6 


RANK  ORDER  OF  MAJOR  STRENGTHS  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  PORTSMOUTH 


NURSING 

POSITION 


SATISFACTION  WITH  WMSN 


FREQUENCY 

ROW  PCT 

COL  PCT 

1 

1 

j  SATISFIED 

NEUTRAL 

DISSATISFIE 

STAFF  NURSE 

1 

41 

1 

64 

1 

36 

♦ 

1 

1 

29.08 

1 

45»39 

1 

25.53 

1 

\ 

64.06 

1 

91.43 

1 

94.74 

1 

CHARGE  NURSE 

i 

17 

1 

4 

1 

2 

♦ 

1 

1 

73.91 

1 

17.39 

1 

8.70 

1 

1 

26.56 

1 

5.71 

1 

5.26 

1 

SUPERVISORY 

1 

6 

1 

2 

1 

0 

♦ 

1 

1 

75.00 

1 

25.00 

1 

0.00 

1 

1 

9.38 

1 

- 

2.86 

1 

0.00 

1 

♦ 

TOTAL 

64 

70 

38 

TOT 

1 


I 


Missing  Cases  =  5 

Chi  Square  =  22.611 

Degrees  of  Freedom  »  4 

Probability'  =  .0002 

Note:  4  cells  have  less  than  5  cases. 


APPENDIX  F-7 


RANK  ORDER  OF  MAJOR  STRENGTHS  OF  THW  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  PORTSMOUTH 


Variable 


Agree 

Frequency  Percent 


Usefulness  As  a  Management  Tool 

65 

36.723X 

Ease  Of  Use 

44 

24.859t 

Takes  Little  Time  To  Do 

34 

19.209X 

Reliable 

30 

16.949S 

Comprehensive 

28 

15.819X 

Accurately  Reflects  Workload 

.  26 

14.689« 

n  =  177  nurses 

(‘tere  than  one  response  could  be  selected) 


APPENDIX  F-d 


RANK  ORDER  OF  MAJOR  WEAKNESSES  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  PERCEIVED  BY  PROFESSIONAL  NURSES  AT  NAVAL  HOSPITAL  PORTSMOUTH 


Agree 

Variable  Frequency  Percent 


Not  Comprehensive 

60 

33.898X 

Inaccurate  In  Reflecting  Workload 

59 

33.333X 

Unreliable 

52 

29.379* 

Takes  Long  Time  To  Do 

46 

25.989* 

Not  Useful  As  A  Management  Tool 

22 

12.429* 

Difficult  To  Use 

6 

3.390* 

n  =  177  nurses 

I  (More  than  one  response  could  be  selected) 

I 

♦ 

I 


I 


I 


APPENDIX  F-9 


NURSING  PERSONNEL  STAFFING  LEVELS  FOR  EACH  SHIFT  AS 
DETERMINED  BY  WORKLOAD  INDEX  CRITERIA  AT 
NAVAL  HOSPITAL  PORTSMOUTH 


A.  RN  STAFF 

Workload  Index  Shifts 

Frequency  Percent 

*Less  than  recomnended  36  16.82t 

**At  recomnended  174  81.31X 

***Greater  than  recomnended  4  1.87X 

214  lOOX 

B.  NRN  STAFF 

Workload  Index  Shifts 

Frequency  Percent 

*Less  than  recomnended  13  6.07X 

**Rleconinended  176  82.24X 

***6reater  than  recomnended  25  11.69X 

214  lOOX 

C.  TOTAL  STAFF 

Workload  Index  Shifts 

Frequency  Percent 

*Less  than  recomnended  39  18.22t 

**Recomnended  159  74.30X 

***Greater  than  recomnended  16  7.48t 

214  lOOX 


Key  to  Levels: 

*Less  than  recomnended: 
♦♦Recommended : 

♦♦♦Greater  than  recomnended: 


F-IO 


Minus  2  persons  or  more 
Minus  1  person  to  plus  1  person 
Plus  2  persons  or  more 


APPENDIX  G-1 


NURSERY  UNITS  AT  SIX  NAVAL  HOSPITALS 


NURSE 

EXPERT 

RATING 


FREQUENCY  I 
PERCENT  I 
ROM  PCT  I 
COL  PCT  I 

I  i 
I 

Category  | 


Category 


Category  I 


Category 


TOTAL 


Category 

I  I 


CHARGE  NURSES  RATING 


Category  Category  Category 
2  I  3  I  4  I 


0  I 
0.00  I 
0.00  I 
0.00  I 

0  I 
0.00  I 
0.00  I 
0.00  I 

2 

7.69 


0  1  10  I 

0.00  }  38.46  I 

0.00  I  100.00  I 
0.00  I  100.00  i 


■  ■■■■■■» 
0  I 
0.00  I 
0.00  I 
0.00  I 


0  i  0  1  II 

0.00  I  0.00  I  3.85  r 
0.00  I  0.00  t  100.00  I 

0.00  I  0.00  I  100.00  I 

>  ■  -  ■ ,  ■  —  I  I.  .  .  4. 

13  10  1 

50.00  38.46  3.85 


TOTAL 


1 

1 

1 

0 

1 

0  1 

0 

1 

1 

1 

3.85 

1 

0.00 

1 

0.00  1 

0.00 

1 

3.85 

1 

100.00 

1 

0.00 

1 

0.00  I 

0.00 

1 

1 

50.00 

1 

0.00- 

1 

0.00  1 

0.00 

1 

1 

1 

1 

13 

I 

0  1 

0 

1 

14 

1 

3.85 

1 

50.00 

1 

0.00  1 

0.00 

1 

53.85 

1 

7.14. 

1 

92.86 

1 

0.00  1 

0.00 

1 

1 

50.00 

1 

100.00 

1 

0.00  1 

0.00 

1 

10 

38.46 


1 

3.85 


26 

100.00 


Percent  of  Agreement  ■  25/26  or  96.I5Z 
Kappa  Statistic  ■  .933 
Standard  Deviation  of  Kappa  ■  .065 
2  Score  ■  14.316 
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APPENDIX  G-2 


NURSE 

EXPERT 

RATING 


POST-PARTUM  UNITS  AT  SIX  NAVAL  HOSPITALS 


FREQUENCY 
PERCENT 
ROM  PCT 


charge  nurses  RATING 


Category  Category  Category  Category 


COL  PCT 

1 

1 

1 

*  — 

2 

1 

3 

1 

4 

1 

A  ^ 

TOTAL 

I 

1 

9 

1 

2 

J 

0 

1 

0 

1 

11 . 

1 

27.27 

1 

6.  06 

1 

0.  00 

1 

0.00 

1 

33.33 

Category 

1 

81.82 

1 

18.18 

1 

0.  00 

1 

0.00 

1 

1 

100.00 

1 

liai 

1 

0.00 

1 

0.00 

1 

"^******** 

• 

*^***^^**' 

•  ▼ 

2 

1 

0 

1 

16 

1 

1 

1 

0 

17 

1 

0.00 

1 

48.48 

1 

3.03 

1 

0.00 

1 

51.52 

Category 

1 

0.00 

1 

94.12 

1 

5.88 

1 

0.00 

1 

» 

0.00 

1 

88.89 

1 

20.00  I 

0.00 

1 

1 

! 

1 

0 

1  0 

1 

4  1 

1 

1 

5 

1 

0.00 

1 

0.00 

1 

12.12 

1 

3.03 

1 

15.15 

Category 

» 

0.00 

1 

0.00 

1 

3  0.00 

1 

20.00 

1 

» 

0.00 

1 

0.00 

1 

80.00 

1 

100.00 

1 

TOTAL 


9 

27,27 


18 

54.55 


5 

15.15 


1 

3.03 


33 

100.00 


Percent  of  Agreement  ■  29/33  or  87.88Z 
Kappa  Statistic  ■  .7996 
Standard  Deviation  of  Kappa  ■  .0938 
Z  Score  ■  8.5177 
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APPENDIX  G-3 


NURSE 

EXPERT 

RATING 


PEDIATRIC  UNITS  AT  SIX  NAVAL  HOSPITALS 


CHARGE  NURSES  RATINGS 


FREQUENCY  1 

PERCENT  K 

ROW  PCT  1  Category 

COL  PCT  1  1 

Category 

1  2  1 

Category 

3 

Category 

»  4 

1 

TOTAL 

1  1  2 

1 

0  t 

0 

I 

0 

1 

2 

1  8.00 

1 

0.00  i 

0.00 

1 

0.00 

1 

8.00 

Category  *  100*00 

1 

0.00  1 

0.00 

1 

0.00 

1 

i  100.00 

1 

0.00  1 

0.00 

1 

0.00 

» 

2  1  0 

1 

4  1 

3 

1 

0 

1 

7 

1  0.00 

1 

16.00  1 

12.00 

1 

0.00 

1 

28.00 

Category  J  0.00 

1 

57.14  1 

42.86 

1 

0.00 

1 

1  0,00 

1 

80.00  1 

20.00 

1 

0.00 

1 

3  1  0 

1 

1  1 

12 

1 

0 

1 

13 

1  0.00 

1 

4.00  1 

48.00 

1 

0.00 

1 

52.00 

Category  1  0.00 

1 

7.69  1 

92.31 

1 

0.00 

1 

1  0.00  1 

20.00  i  80.00  1 

0.00 

1 

I  0  i 

0  1  0  1 

3 

1 

3 

1  0.00 

1 

0.00  1 

0.00 

1 

12.00 

1 

12.00 

Category  1  0.00 

1 

0.00  1 

0.00 

1 

100.00 

1 

1  0.00 

1 

0.00  1 

0.00 

1 

100.00 

1 

TOTAL  2 

5 

15 

3 

25 

8.00 

20.00 

60.00 

12.00 

100.00 

Percent  of  Agreement  >  21/25  or  84Z 

Kappa  Statistic  •  .738 

Standard  Deviation  of  Kappa  -  .119 


G-4 


2  Score  -  6.1537 


APPENDIX  G-4 


NURSE 

EXPERT 

RATING 


ICU/CCU  UNITS  AT  SIX  NAVAL  HOSPITALS 


CHARGE  NURSES  RATING 


FREQUENCY 
PERCENT 
ROW  PCT 


Category  Category  Category  Category 


COL  PCT 

1 

2 

1 

3 

1 

4 

1 

5 

1 

TOTAL 

2 

1 

1 

1 

0 

1 

0 

1 

0 

1 

1 

1 

5.56 

1 

0.00 

1 

0.00 

1 

0.00 

1 

5.56 

Category 

1 

100.00 

1 

0.90 

1 

0.00 

1 

0.00 

1 

1 

50.00 

1 

6.00 

1 

0.00  1 

0.00 

1 

3 

1 

1 

1 

5 

1 

0 

1 

0 

1 

6 

1 

5,.  5  6 

1 

27.78 

1 

0.00 

1 

0.00 

1 

33.33 

Category 

1 

16.67 

1 

83.33 

1 

0.00 

1 

0.00 

1 

1 

50.00 

1 

100.00 

1 

0.  00 

1 

0.00 

1 

1  III 

II  1  III  III  III  II  T 

1  II  ■  ■  ■ 

4 

1 

0 

1 

0 

1 

6 

1 

0 

1 

6 

1 

0.00 

1 

0.00 

1 

33.33 

1 

0.00 

1 

33.33 

Category 

1 

0.00 

1 

0.90 

1  100.00 

1 

0.00 

» 

1 

0.00 

1 

0.00 

1  100.00  1 

0.00 

1 

5 

1 

0 

1 

0 

1 

0  1 

5 

1 

5 

1 

0.00 

1 

0.00 

1 

0.00 

1 

27.78 

1 

27.78 

Category 

1 

0.00 

1 

0.00 

1 

0.00 

1  100.00 

1 

1 

0.00 

1 

0.00 

1 

0.00  t  100.00 

1 

TOTAL 

2 

5 

6 

5 

18 

ll.ll 

27.73 

33.33 

27.78 

100.00 

Percent  of  Agreement  ■  17/18  or  94.44S 

Kappa  Statistic  •  .922 

Standard  Deviation  of  Kappa  -  .0757 


2  Score  •  12.176 


APPENDIX  G-5 


SURGICAL  UNITS  AT  SIX  NAVAL  HOSPITALS 


CHARGE  NURSES  RATING 


NURSE 

EXPERT 

RATING 


FREQUENCY 
PERCENT 
ROW  PCT 
COL  PCT 


Category 


Category 


Category 


Category 


TOTAL 


Category 

1 

15 

2a«as 

93.75 

93.75 

1 

1.92 

4.17 

6.25 


Category  Category  Category  Category 
I  2  1  3  1  4  1  5 


0 

0.00 
0.00 
0.  00 

0 

0.00 

0.00 

0.00 

16 

30.77 


I  I 
1.92  I 
6.25  I 
4.35  i 


0  1  0  1  0 

0.00  I  0.00  i  0.00 

0.00  I  0.00  I  0.00 

0.00  I  0.00  I  0.00 

2  1  0  1  0 

3.85  I  0.00  1  0.00 

a. 33  (  0.00  I  0.00 

.8.18  I  0.00  I  0.00 

9  1  0  (  0 

L7.31  I  0.00  I  0.00 

>0.00  i  0.00  I  0.00 

11.82  I  0.00  I  0.00 

0  )  It  1 

0.00  I  1.92  I  1.92 

0.00  I  50.00  I  50.00 

0.00  I  100.00  I  100.00 


0  I 
0.00  i 
0.00  I 
0.00  I 


I  21  I  21 

I  40.38  I  3.85  I 

I  87.50  I  8.33  ( 

I  91.30  I  18.18  I 


II  9  1 

1.92  I  17.31  I 
10.00  I  90.00  i 
4.35  I  81.82  I 


0  I 
0.00  I 
0.00  I 
0.00  I 

23 

44.23 


0  1  It 

0.00  I  1.92  I 
0.00  I  50.00  I 


0  I 

0.00  1 
0.00  I 
0.00  I 

mm  m  wm  —  ^ 

0  I 
0.00  I 
0.00  I 
0.00  I 

1  t 
1.92  I 


II 

21.15 


I 

1.92 


1 

1.92 


TOTAL 

16 

30.77 


24 

46.15 


10 

19.23 


2 

3.85 


52 

100.00 


Percent  of  Agreement  •  46/52  or  88.462 

Kappa  Statistic  ■  .8846 

Standard  Deviation  of  Kappa  -  .067 


Z  Score  -  12.287 


appendix  G-6 


MEDICAL  UNITS  AT  SIX  NAVAL  HOSPITALS 


CHARGE  NURSES  RATING 


NURSE 

EXPERT 

RATING 

FRE0U6MCY! 
PERCENT  1 
ROW  PCT  \  ( 
COL  PCT  1 

Category 

1 

Category 
t  2 

Category 

1  3 

1 

Category 

4 

1 

1  1 

15 

1 

6 

1 

0 

1 

0 

1 

20.00 

1 

8.00 

1 

0.00 

1 

0.00 

Category  i 

71.43 

1 

28.57 

1 

0.00 

1 

0.00 

1 

93.75 

1 

16.22 

1 

0.  00 

1 

0.00 

2  1 

1 

1 

30 

1 

9 

1 

0 

1.33 

1 

40.00 

1 

12.00 

1 

0.00 

Category  1 

2.50 

1 

75.00 

1 

22.50 

1 

0.00 

1 

6.25 

1 

81.08 

1 

45.00 

1 

0.00 

3  1 

0 

1 

1 

1 

11 

1 

1 

1 

0.00 

1 

1.33 

J 

14.67 

1 

1.33 

Category  1 

0.00 

1 

7.59 

1 

84.62 

J 

7.69 

1 

0.00 

1 

2.70 

1 

55.00 

1 

50.00 

4  1 

0 

1 

0 

1 

0 

1 

1 

1 

0.00 

1 

0.00 

1 

0.00 

1 

1.33 

Category  1 

0.00 

1 

0.00 

1 

0.00 

1 

100.00 

1 

0.00 

1 

0.00 

1 

0.00 

\ 

50.00 

TOTAL 

16 

37 

20 

2 

21.33 

49.33 

26.67 

2.67 

Percent  of  Agreement  -  57/75  or  76Z 

Rappa  Statistic  *  .619 

Standard  Deviation  of  Kappa  ■  .078 


G-7 


TOTAL 

21 

28.00 


40 

53.33 


13 

17.33 


1 

1.33 


75 

100.00 


S  Score  ■  7.920 
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